File: //lib/python3/dist-packages/chardet/__pycache__/charsetprober.cpython-312.pyc
�
-��_� � �6 � d dl Z d dlZddlmZ G d� de� Zy)� N� )� ProbingStatec �z � e Zd ZdZd d�Zd� Zed� � Zd� Zed� � Z d� Z
e d � � Z e d
� � Z
e d � � Zy)
�
CharSetProbergffffff�?Nc �\ � d | _ || _ t j t � | _ y �N)�_state� lang_filter�logging� getLogger�__name__�logger)�selfr
s �7/usr/lib/python3/dist-packages/chardet/charsetprober.py�__init__zCharSetProber.__init__' s# � ��� �&��� �'�'��1�� � c �. � t j | _ y r )r � DETECTINGr �r s r �resetzCharSetProber.reset, s � �"�,�,�� r c � � y r � r s r � charset_namezCharSetProber.charset_name/ s � �r c � � y r r )r �bufs r �feedzCharSetProber.feed3 s � � r c � � | j S r )r r s r �statezCharSetProber.state6 s � ��{�{�r c � � y)Ng r r s r �get_confidencez CharSetProber.get_confidence: s � �r c �4 � t j dd| � } | S )Ns ([ -])+� )�re�sub)r s r �filter_high_byte_onlyz#CharSetProber.filter_high_byte_only= s � ��f�f�&��c�2���
r c �� � t � }t j d| � }|D ]C }|j |dd � |dd }|j � s|dk rd}|j |� �E |S )u9
We define three types of bytes:
alphabet: english alphabets [a-zA-Z]
international: international characters [-ÿ]
marker: everything else [^a-zA-Z-ÿ]
The input buffer can be thought to contain a series of words delimited
by markers. This function works to filter all words that contain at
least one international character. All contiguous sequences of markers
are replaced by a single space ascii character.
This filter applies to all scripts which do not use English characters.
s% [a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N���� �r" )� bytearrayr# �findall�extend�isalpha)r �filtered�words�word� last_chars r �filter_international_wordsz(CharSetProber.filter_international_wordsB sw � � �;��
�
�
�O� � �� �D� �O�O�D��"�I� &� �R�S� �I� �$�$�&�9�w�+>� � � �O�O�I� &� � �r c �> � t � }d}d}t t | � � D ]a }| ||dz }|dk( rd}n|dk( rd}|dk s� |j � r�1||kD r'|s%|j | || � |j d� |dz }�c |s|j | |d � |S )
a�
Returns a copy of ``buf`` that retains only the sequences of English
alphabet and high byte characters that are not between <> characters.
Also retains English alphabet and high byte characters immediately
before occurrences of >.
This filter can be applied to all scripts which contain both English
characters and extended ASCII characters, but is currently only used by
``Latin1Prober``.
Fr r � >� <Tr( r" N)r) �range�lenr, r+ )r r- �in_tag�prev�curr�buf_chars r �filter_with_english_lettersz)CharSetProber.filter_with_english_lettersg s� � � �;�������#�c�(�O�D��4��q��)�H��4� � ���T�!� �� �'�!�(�*:�*:�*<��$�;�v� �O�O�C��T�N�3� �O�O�D�)��a�x��% $�* �
�O�O�C���J� '��r r )r
�
__module__� __qualname__�SHORTCUT_THRESHOLDr r �propertyr r r r � staticmethodr% r1 r; r r r r r # s� � � ��2�
-� �� ��
� �� ��� �� �� �"� �"�H �)� �)r r )r r# �enumsr �objectr r r r �<module>rC s ��: � � �n�F� nr