$50 GRAYBYTE WORDPRESS FILE MANAGER $27

B

<_9Y��@s0dd
lZdd
l
Z
ddlmZ
Gdd�de�Zd
S)�N�
)
�ProbingStatec@sneZ
dZd
Zddd�
Zdd�Zedd��
Zd	d
�Zedd��
Z	d
d�Z
edd��
Zedd��
Z
edd��
ZdS)�
CharSetProbergffffff�?NcCsd|_|
|_
t�t�
|_dS)
N)�_state�lang_filter�logging�	getLogger�__name__�logger)�selfr�r�F/opt/alt/python37/lib/python3.7/site-packages/chardet/charsetprober.py�__init__'s


zCharSetProber.__init__c

Cstj
|_dS)
N)rZ	DETECTINGr)
rrrr
�reset,s
zCharSetProber.resetc


CsdS)
Nr)
rrrr
�charset_name/szCharSetProber.charset_namec
CsdS)
Nr)r�bufrrr
�feed3s
zCharSetProber.feedc


Cs|jS)
N)
r)
rrrr
�state6szCharSetProber.statec


Csd
S)Ngr)
rrrr
�get_confidence:s
zCharSetProber.get_confidencec

Cst�
d
d|�}|S)Ns([-])+�
 )�re�sub)
rrrr
�filter_high_byte_only=s
z#CharSetProber.filter_high_byte_onlyc
Cs`t�}
t
�d
|�}xH|D]@}|
�|dd��

|dd�}|��sN|dkrNd}|
�|�

qW|
S)u9
        We define three types of bytes:
        alphabet: english alphabets [a-zA-Z]
        international: international characters [€-ÿ]
        marker: everything else [^a-zA-Z€-ÿ]

        The input buffer can be thought to contain a series of words delimited
        by markers. This function works to filter all words that contain at
        least one international character. All contiguous sequences of markers
        are replaced by a single space ascii character.

        This filter applies to all scripts which do not use English characters.
        s%[a-zA-Z]*[�-�]+[a-zA-Z]*[^a-zA-Z�-�]?N����
�r)�	bytearrayr�findall�extend�isalpha)r�filtered�words�wordZ	last_charrrr
�filter_international_wordsBs





z(CharSetProber.filter_international_wordsc
Cs�t�}
d
}d}x~t
t|�
�
D]n}|||d�}|dkr>d
}n|dkrJd}|dkr|��s||kr�|s�|
�|||��

|
�d�

|d}qW|s�|
�||d	��

|
S)
a�

        Returns a copy of ``buf`` that retains only the sequences of English
        alphabet and high byte characters that are not between <> characters.
        Also retains English alphabet and high byte characters immediately
        before occurrences of >.

        This filter can be applied to all scripts which contain both English
        characters and extended ASCII characters, but is currently only used by
        ``Latin1Prober``.
        Fr
r�
>�
<TrrN)r�range�lenrr)rrZin_tag�prev�currZbuf_charrrr
�filter_with_english_lettersgs"






z)CharSetProber.filter_with_english_letters)
N)r	�
__module__�__qualname__ZSHORTCUT_THRESHOLDrr�propertyrrrr�staticmethodrr"r)rrrr
r#s
%r)rrZenumsr�objectrrrrr
�<module>s