HEX
Server: Apache/2.4.52 (Ubuntu)
System: Linux spn-python 5.15.0-89-generic #99-Ubuntu SMP Mon Oct 30 20:42:41 UTC 2023 x86_64
User: arjun (1000)
PHP: 8.1.2-1ubuntu2.20
Disabled: NONE
$ pwd: /usr/lib/python3/dist-packages/sos/cleaner/__pycache__/
Upload Files
File: //usr/lib/python3/dist-packages/sos/cleaner/__pycache__/__init__.cpython-310.pyc
o

i��d���@s$ddlZddlZddlZddlZddlZddlZddlZddlm	Z	ddl
m
Z
ddlmZddlm
Z
ddlmZddlmZddlmZdd	lmZdd
lmZddlmZddlmZdd
lmZmZmZm Z ddl!m"Z"m#Z#ddl$m%Z%ddl&m'Z'm(Z(m)Z)ddl*m+Z+Gdd�de�Z,dS)�N)�ThreadPoolExecutor)�datetime)�getpwuid)�__version__)�SoSComponent)�SoSIPParser)�SoSMacParser)�SoSHostnameParser)�SoSKeywordParser)�SoSUsernameParser)�
SoSIPv6Parser)�SoSReportArchive�SoSReportDirectory�SoSCollectorArchive�SoSCollectorDirectory)�DataDirArchive�TarballArchive)�InsightsArchive)�get_human_readable�
import_module�ImporterHelper)�fillcsdeZdZdZdZdggdgdddddgd	�Z		dN�fd
d�	ZdOdd
�ZdOdd�ZdOdd�Z	dOdd�Z
dd�Zedd��Z
dd�Zdd�Zedd��Zdd�Zd d!�Zd"d#�Zd$d%�Zd&d'�Zd(d)�Zd*d+�Zd,d-�Zd.d/�ZdPd0d1�Zd2d3�Zd4d5�Zd6d7�Zd8d9�Zd:d;�Zd<d=�Z d>d?�Z!d@dA�Z"dQdBdC�Z#dDdE�Z$dFdG�Z%dHdI�Z&dOdJdK�Z'dLdM�Z(�Z)S)R�
SoSCleanera
    This function is designed to obfuscate potentially sensitive information
    from an sos report archive in a consistent and reproducible manner.

    It may either be invoked during the creation of a report by using the
    --clean option in the report command, or may be used on an already existing
    archive by way of 'sos clean'.

    The target of obfuscation are items such as IP addresses, MAC addresses,
    hostnames, usernames, and also keywords provided by users via the
    --keywords and/or --keyword-file options.

    For every collection made in a report the collection is parsed for such
    items, and when items are found SoS will generate an obfuscated replacement
    for it, and in all places that item is found replace the text with the
    obfuscated replacement mapped to it. These mappings are saved locally so
    that future iterations will maintain the same consistent obfuscation
    pairing.

    In the case of IP addresses, support is for IPv4 and IPv6 - effort is made
    to keep network topology intact so that later analysis is as accurate and
    easily understandable as possible. If an IP address is encountered that we
    cannot determine the netmask for, a random IP address is used instead.

    For IPv6, note that IPv4-mapped addresses, e.g. ::ffff:10.11.12.13, are
    NOT supported currently, and will remain unobfuscated.

    For hostnames, domains are obfuscated as whole units, leaving the TLD in
    place.

    For instance, 'example.com' may be obfuscated to 'obfuscateddomain0.com'
    and 'foo.example.com' may end up being 'obfuscateddomain1.com'.

    Users will be notified of a 'mapping' file that records all items and the
    obfuscated counterpart mapped to them for ease of reference later on. This
    file should be kept private.
    z6Obfuscate sensitive networking information in a report�auto�N� /etc/sos/cleaner/default_mappingF�)�archive_type�domains�disable_parsers�jobs�keywords�keyword_file�map_file�	no_update�keep_binary_files�target�	usernamesc	s�|stt|��|||�d|_nC|d|_|d|_|d|_|d|_|d|_d|_t	|jd�s6d	|j_
d
|j_t�
d�|_t�
d�|_tjtj�|jd
�dd�|��|��|_t�d�||_|j��|_|jj�d
�|_t|j�t|j�t |j�t!|j�t"|j�t#|j�g|_$|jj%D]1}|j$D]+}|j&�'��(d�d�)�}|�'��)�|kr�|�*d|�|j�+d|�|j$�,|�q�q�t-t.t/t0t1t2t3g|_4d|_5|�*d|j�dS)NT�options�tmpdir�sys_tmp�policy�manifestFr rr�sos�sos_ui�cleaner��exist_ok�?�parserrzDisabling parser: %szaDisabling the '%s' parser. Be aware that this may leave sensitive plain-text data in the archive.z%Cleaner initialized. From cmdline: %s)6�superr�__init__�from_cmdline�optsr)r*r+r,�hasattrr r�logging�	getLogger�soslog�ui_log�os�makedirs�path�join�validate_parser_values�
load_map_file�cleaner_mapping�umask�in_place�get_preferred_hash_name�	hash_name�
components�add_section�
cleaner_mdr	rrrr
r�parsersr�name�lower�split�strip�log_info�warning�removerr
rrrrr�
archive_types�nested_archive)	�selfr3�args�cmdlinerE�hook_commons�_parser�_loaded�_loaded_name��	__class__��6/usr/lib/python3/dist-packages/sos/cleaner/__init__.pyr5^sj






�	
�����

�zSoSCleaner.__init__cCsd|r
d||fSd|fS)Nz[cleaner%s] %sz:%srr^�rU�msg�callerr^r^r_�_fmt_log_msg�szSoSCleaner._fmt_log_msgcC�|j�|�||��dS�N)r;�debugrcr`r^r^r_�	log_debug��zSoSCleaner.log_debugcCrdre)r;�inforcr`r^r^r_rP�rhzSoSCleaner.log_infocCrdre)r;�errorrcr`r^r^r_�	log_error�rhzSoSCleaner.log_errorcCs0d}d}|��D]
}|t||dd�d}q|S)N�PrF)�replace_whitespace�
)�
splitlinesr)rUra�width�_fmt�liner^r^r_�_fmt_msg�s
zSoSCleaner._fmt_msgcCs|�d�|�|j�dS)NzSoS Cleaner Detailed Help)�	set_title�add_text�__doc__)�cls�sectionr^r^r_�display_help�s
zSoSCleaner.display_helpcCsi}d}tj�|jj�rtd|jj��tj�|jj�s-|jj|kr+|�d|jj�|St|jjd��L}zt	�
|�}Wn+t	jyK|�d�Yn%tyg}z|�d|jj|f�WYd}~nd}~wwWd�|SWd�|SWd�|S1s�wY|S)z�Verifies that the map file exists and has usable content.

        If the provided map file does not exist, or it is empty, we will print
        a warning and continue on with cleaning building a fresh map
        rz$Requested map file %s is a directoryzHERROR: map file %s does not exist, will not load any obfuscation matches�rzOERROR: Unable to parse map file, json is malformed. Will not load any mappings.zERROR: Could not load '%s': %sN)r=r?�isdirr7r#�	Exception�existsrk�open�json�load�JSONDecodeError)rU�_conf�default_map�mf�errr^r^r_rB�sH���
�
����
�	�
�	�
�	�	zSoSCleaner.load_map_filec
Cs�|�d�}|j�dt�|j�|�|jjsLztd�WdSty3|j�d�|�d�YdSt	yK}z
|�d|�WYd}~dSd}~wwdS)z�When we are directly running `sos clean`, rather than hooking into
        SoSCleaner via report or collect, print a disclaimer banner
        a�This command will attempt to obfuscate information that is generally considered to be potentially sensitive. Such information includes IP addresses, MAC addresses, domain names, and any user-provided keywords.

Note that this utility provides a best-effort approach to data obfuscation, but it does not guarantee that such obfuscation provides complete coverage of all such data in the archive, or that any obfuscation is provided to data that does not fit the description above.

Users should review any resulting data and/or archives generated or processed by this utility for remaining sensitive content before being passed to a third party.
z
sos clean (version %s)
z-
Press ENTER to continue, or CTRL-C to quit.
z
Exiting on user cancel��N)
rsr<rirr7�batch�input�KeyboardInterrupt�_exitr|)rUra�er^r^r_�print_disclaimer�s
���zSoSCleaner.print_disclaimercCs�d|_|�dd�}|jdddd�|jdd	gd
�dd�|jd
dgdd�|jddgddd�|jdddtdd�|jddgddd�|jddddd �|jd!d"d#d$d%�|jd&d'd(d)d*d+�|jd,d(d)d-d.d/�|jd0d1gdd2d+�dS)3Nzsos clean|mask TARGET [options]zCleaner/Masking Optionsz7These options control how data obfuscation is performedr&�TARGETz%The directory or archive to obfuscate)�metavar�helpz--archive-typer)r�report�collect�insightszdata-dir�tarballz8Specify what kind of archive the target was generated as)�default�choicesr�z	--domains�extendz!List of domain names to obfuscate)�actionr�r�z--disable-parsersrzCDisable specific parsers, so that those elements are not obfuscated)r�r��destr�z-jz--jobsrz&Number of concurrent archives to clean)r��typer�z
--keywordsr!zList of keywords to obfuscatez--keyword-filer"z&Provide a file a keywords to obfuscate)r�r�r�z
--map-filer#rz;Provide a previously generated mapping file for obfuscation)r�r�r�z--no-updater$F�
store_truez<Do not update the --map-file with new mappings from this run)r�r�r�r�z--keep-binary-filesr%zGKeep unprocessable binary files in the archive instead of removing them)r�r�r�r�z--usernamesr'zList of usernames to obfuscate)�usage�add_argument_group�add_argument�int)rwr3�	clean_grpr^r^r_�add_parser_options�s^���
���
���
��

�zSoSCleaner.add_parser_optionscCs||j_dS)z�For use by report and collect to set the TARGET option appropriately
        so that execute() can be called just as if we were running `sos clean`
        directly from the cmdline.
        N)r7r&)rUr?r^r^r_�set_target_path"szSoSCleaner.set_target_pathcCs�d}|jjdkr$|jj�dd�}|jD]}|j|kr"||jj|j�}qn|jD]}|�|jj�r:||jj|j�}nq'|s?dS|j�	|�|j
rY|j�|���|j�
|�||_|jrd|jj|j_dSdS)z�The target path is not a directory, so inspect it for being an
        archive or an archive of archives.

        In the event the target path is not an archive, abort.
        Nr�-�_)r7r�replacerS�	type_namer&r)�
check_is_type�report_paths�append�	is_nestedr��get_nested_archivesrRrT�description�ui_name)rU�_arc�
check_type�archive�arcr^r^r_�inspect_target_archive)s.

��
��z!SoSCleaner.inspect_target_archivecCs4|jjD]}t|�d��dkrtd|�d���qdS)z�Check any values passed to the parsers via the commandline, e.g.
        the --domains option, to ensure that they are valid for the parser in
        question.
        �.�zInvalid value 'z0' given: --domains values must be actual domainsN)r7r�lenrNr|)rU�_domr^r^r_rAFs
���z!SoSCleaner.validate_parser_valuescCs�|jj�d�d�d�d|_|jr|��g|_tj�	|jj�s0|j
�d|jj�|�d�|�
�|jsB|j
�d�|�d�g|_|jD]}|jdkrT|j��qH|��|��|��|jst|jrid	S|j
�d
�|�d�|j
�dt|j��|��}|�|�}|�|�|��|jr�dd
�|jD�}||fSd	}t|j�dkr�|��}nE|jd}|j}|�|j�}|d	ur�|� d|�d�d|j!f�}	t"tj�#|j$|	�d��
}
|
�%|�Wd	�n1s�wY|�&�tj�#|j$|� |�d�d��}t'�(||�t�)|�}|j
�d|���|j
�d|�d��|j
�dt*|j+����|j
�dt,|j-�j.�d��|j
�d�|�/�d	S)a,SoSCleaner will begin by inspecting the TARGET option to determine
        if it is a directory, archive, or archive of archives.

        In the case of a directory, the default behavior will be to edit the
        data in place. For an archive will we unpack the archive, iterate
        over the contents, and then repack the archive. In the case of an
        archive of archives, such as one from SoSCollector, each archive will
        be unpacked, cleaned, and repacked and the final top-level archive will
        then be repacked as well.
        �/���z.tarrz,Invalid target: no such file or directory %sr�z'No valid archives or directories found
zHostname ParserNz#No reports obfuscated, aborting...
z&
Successfully obfuscated %s report(s)
cSsg|]}|j�qSr^)�final_archive_path)�.0�ar^r^r_�
<listcomp>�sz&SoSCleaner.execute.<locals>.<listcomp>z%s.%s�wz2A mapping of obfuscated elements is available at
	z)
The obfuscated archive is available at
	rnz	Size	z	Owner	zcPlease send the obfuscated archive to your support representative and keep the mapping file private)0r7r&rN�arc_namer6r�r�r=r?r}r<rjr�r��completed_reportsrKrL�mapping�set_initial_counts�preload_all_archives_into_maps�generate_parser_item_regexes�obfuscate_report_pathsrErir��compile_mapping_dict�write_map_for_archive�write_map_for_config�write_stats_to_manifest�rebuild_nested_archiver��get_new_checksum�obfuscate_stringrGr~r@r*�write�write_cleaner_log�shutil�move�statr�st_sizer�st_uid�pw_name�cleanup)rUr3�_map�map_path�	arc_paths�
final_path�arc_pathr��checksum�chksum_name�cf�arcstatr^r^r_�executeRs��




�
�



���
�
�zSoSCleaner.executecCs�|jd}|j|d�|jD]#}|j�d�d}|�|j�}|dur1d||jf}|jj||d�qt	�
|jj�D],\}}}|D]$}	t	j
�||	�}
|
�|jj�d}|�d�}|jj|
|d�t	�|
�q@q9|jdd	�|j�|jj�S)
z�Handles repacking the nested tarball, now containing only obfuscated
        copies of the reports, log files, manifest, etc...
        z-obfuscated)rLr�r�Nzchecksums/%s.%s�r�T)r�)r��
setup_archiver�r�rNr�rGr��
add_stringr=�walkrT�extracted_pathr?r@�lstrip�add_filerRr��finalizer7�compression_type)rUr�r��arc_destr��dname�dirn�dirs�files�filename�fnamer^r^r_r��s&

�
�z!SoSCleaner.rebuild_nested_archivecCs2i}|jD]}i||j<||j�|���q|S)aBuild a dict that contains each parser's map as a key, with the
        contents as that key's value. This will then be written to disk in the
        same directory as the obfuscated report so that sysadmins have a way
        to 'decode' the obfuscation locally
        )rK�map_file_key�update�get_map_contents)rUr�r3r^r^r_r��s


zSoSCleaner.compile_mapping_dictcCsDt|d��}|�tj|dd��Wd�|S1swY|S)zjWrite the mapping to a file on disk that is in the same location as
        the final archive(s).
        r�r)�indentN)r~r�r�dumps)rUr�r?r�r^r^r_�write_map_to_file�s
��zSoSCleaner.write_map_to_filec
Cs`ztj�|j|�d|j��}|�||�WSty/}z|�d|�WYd}~dSd}~ww)Nz%s-private_mapz$Could not write private map file: %s)	r=r?r@r*r�r�r�r|rk)rUr�r�r�r^r^r_r��s���z SoSCleaner.write_map_for_archivec
Cs�|jjrG|jjsItj�|jj�}	ztj|dd�|�||jj�|�d|jj�WdSt	yF}z|�
d|�WYd}~dSd}~wwdSdS)z}Write the mapping to the config file so that subsequent runs are
        able to provide the same consistent mapping
        Tr0zWrote mapping to %sz(Could not update mapping config file: %sN)r7r#r$r=r?�dirnamer>r�rgr|rk)rUr��cleaner_dirr�r^r^r_r��s����zSoSCleaner.write_map_for_configcCs�tj�|jd|j�}t|d��}|j�d�|j��D]}|�	|�qWd�n1s.wY|rD|�
|�|jj|dd�dSdS)z�When invoked via the command line, the logging from SoSCleaner will
        not be added to the archive(s) it processes, so we need to write it
        separately to disk
        z%s-obfuscation.logr�rNzsos_logs/cleaner.logr�)
r=r?r@r*r�r~�sos_log_file�seek�	readlinesr��obfuscate_filer�r�)rUr��log_name�logfilerrr^r^r_r��s���
�zSoSCleaner.write_cleaner_logc
Cs�z'd}t|d�}t�|j�}	|�|�}|sn|�|�q|��|��dWStyA}z|�	d|�WYd}~dSd}~ww)zvCalculate a new checksum for the obfuscated archive, as the previous
        checksum will no longer be valid
        i�rbTrnz#Could not generate new checksum: %sN)
r~�hashlib�newrG�readr��close�	hexdigestr|rg)rU�archive_path�	hash_size�
archive_fp�digest�hashdatar�r^r^r_r�
s"


���zSoSCleaner.get_new_checksumcCs�zDdt|j�|jjf}|j�|�|jjr|j�d�t|jj�}|j	|j
|jdd�|jdd�|jrB|�
�|�
|j�Wd	SWd	StyY|j�d�t�d�Yd	Sw)
z�Perform the obfuscation for each archive or sos directory discovered
        during setup.

        Each archive is handled in a separate thread, up to self.opts.jobs will
        be obfuscated concurrently.
        zFFound %s total reports to obfuscate, processing up to %s concurrently
zpWARNING: binary files that potentially contain sensitive information will NOT be removed from the final archive
r�)�	chunksizeT)�waitzExiting on user cancelr�N)r�r�r7r r<rir%rQr�map�obfuscate_report�shutdownrT�_replace_obfuscated_archivesr�r=r�)rUra�poolr^r^r_r�s*�����z!SoSCleaner.obfuscate_report_pathscCsV|jD]%}t�|j�|jj}|j�d�d}tj�	||�}t
�|j|�||_qdS)z�When we have a nested archive, we need to rebuild the original
        archive, which entails replacing the existing archives with their
        obfuscated counterparts
        r�r�N)r�r=rRrrTr�r�rNr?r@r�r�)rUr�r�r��	dest_namer^r^r_r
:s
�z'SoSCleaner._replace_obfuscated_archivescCs|jD]}|��qdS)z�For the parsers that use prebuilt lists of items, generate those
        regexes now since all the parsers should be preloaded by the archive(s)
        as well as being handed cmdline options and mapping file configuration.
        N)rK�generate_item_regexes)rUr3r^r^r_r�Gs

�z'SoSCleaner.generate_parser_item_regexescCs|jD]�}|j����d��}|�||�D]G}|�|�}|s q|�d|�d|�d|j���|�	�D])}z|�
|�Wq3ty\}z|�d|�d|�d|���WYd}~q3d}~wwq|�||�}	|	r}|�d|�d	|j���|	D]}
|j
�|
�qt|j|D]}|j
�|�q�qdS)
a*
        For each archive we've determined we need to operate on, pass it to
        each prepper so that we can extract necessary files and/or items for
        direct regex replacement. Preppers define these methods per parser,
        so it is possible that a single prepper will read the same file for
        different parsers/mappings. This is preferable to the alternative of
        building up monolithic lists of file paths, as we'd still need to
        manipulate these on a per-archive basis.

        :param archive: The archive we are currently using to prepare our
                        mappings with
        :type archive:  ``SoSObfuscationArchive`` subclass

        :param prepper: The individual prepper we're using to source items
        :type prepper:  ``SoSPrepper`` subclass
        rz	Prepping z parser with file z from zFailed to prep z
 map from z: Nz mapping with items from )rKrLrMrNrO�get_parser_file_list�get_file_contentrgr�ro�
parse_liner|�get_items_for_mapr��add�regex_items�add_regex_item)rUr��prepperrY�pname�_file�contentrrr��	map_items�item�ritemr^r^r_�_prepare_archive_with_prepperOs<

��������z(SoSCleaner._prepare_archive_with_prepperccs\�ttjj�}g}|��D]}|�td|����q
t|dd�d�D]	}||jd�Vq"dS)a
        Discover all locally available preppers so that we can prepare the
        mappings with obfuscation matches in a controlled manner

        :returns: All preppers that can be leveraged locally
        :rtype:   A generator of `SoSPrepper` items
        zsos.cleaner.preppers.cSs|jSre)�priority)�xr^r^r_�<lambda>�sz)SoSCleaner.get_preppers.<locals>.<lambda>)�key)r(N)	rr-r/�preppers�get_modulesr�r�sortedr7)rU�helper�preps�_preprr^r^r_�get_preppersys��zSoSCleaner.get_prepperscCs4|�d�|��D]}|jD]}|�||�qq	dS)aBefore doing the actual obfuscation, if we have multiple archives
        to obfuscate then we need to preload each of them into the mappings
        to ensure that node1 is obfuscated in node2 as well as node2 being
        obfuscated in node1's archive.
        z.Pre-loading all archives into obfuscation mapsN)rPr*r�r)rUrr�r^r^r_r��s

��z)SoSCleaner.preload_all_archives_into_mapscCs��z<|j�|j�}t��}|�d|�|js|��|�d�|�	�D]N}|�
|jd�d}|�|�r5q#|jj
sD|�|�rD|�|�q#z|�|||j�}|rU|�||�Wq#tyq}z|�d||f�WYd}~q#d}~wwz|�|�Wnty�}z|jd||jd�WYd}~nd}~wwz|�|�Wnty�}z|jd	||jd�WYd}~nd}~ww|j�s|��}|r�|�d
�z|�|�|j��|�|�Wn%ty�}z|�d|j|f�|�d|�WYd}~WdSd}~ww|j�|�t��}	|�d
|	�|�d|	|�|�dt|j��|�d|j �d}
|j!�r4d}
|
|j!}
|�d|
�WdSt�y\}z|j"�#d|j|f�WYd}~dSd}~ww)z�Individually handle each archive or directory we've discovered by
        running through each file therein.

        Positional arguments:

            :param report str:      Filepath to the directory or archive
        �
start_timezBeginning obfuscation...r�r�zUnable to parse file %s: %sNz#Failed to obfuscate directories: %s�rbz Failed to obfuscate symlinks: %szRe-compressing...z!Archive %s failed to compress: %sz!Failed to re-compress archive: %s�end_time�run_time�files_obfuscated�total_substitutionsrz! [removed %s unprocessable files]zObfuscation completed%sz!Exception while processing %s: %s)$rJrI�archive_namer�now�	add_field�is_extracted�extract�
report_msg�
get_file_listrN�should_skip_filer7r%�should_remove_file�remove_filer��update_sub_countr|rg�obfuscate_directory_namesrP�obfuscate_symlinksr��get_compression�rename_top_dirr��compressr�r�r��
file_sub_list�total_sub_count�removed_file_countr<ri)rUr��arc_mdr+r��
short_name�countr��methodr-�rmsgr^r^r_r�s�

�
�����
���
���

�����
���zSoSCleaner.obfuscate_reportcs�|sdSd}�s|�d�d�tj�|�s�|jd�p||d�tjd|jd�}�fd	d
�|jD�}t	|ddd
��=}|D]2}z|�
||�\}}	||	7}|�|�Wq<tyn}
z|jd�|
f|d�WYd}
~
q<d}
~
wwWd�n1sywY|�
d�|r�t�|j|�|��|���d�d�}����d�d|�}|�kr�|���d}
tj�|
|�}tj�|�s�t�||�|S|�t�|��}t�|�t�||�|S)a5Obfuscate and individual file, line by line.

        Lines processed, even if no substitutions occur, are then written to a
        temp file without our own tmpdir. Once the file has been completely
        iterated through, if there have been substitutions then the temp file
        overwrites the original file. If there are no substitutions, then the
        original file is left in place.

        Positional arguments:

            :param filename str:        Filename relative to the extracted
                                        archive root
        Nrr�r�zObfuscating %sr,r�)�mode�dircs(g|]}t�fdd�|jD��s|�qS)c3s�|]}|���VqdSre)�match)r��_skip�rEr^r_�	<genexpr>�s�

�z7SoSCleaner.obfuscate_file.<locals>.<listcomp>.<genexpr>)�any�
skip_patterns)r��_prMr^r_r��s���z-SoSCleaner.obfuscate_file.<locals>.<listcomp>rzr�)�errorszUnable to obfuscate %s: %s)rNr=r?�islinkrg�tempfile�NamedTemporaryFiler)rKr~�obfuscate_liner�r|r�r��copyfilerLrr�r�r@�rename�readlinkrR�symlink)rUr�rEr��subs�tfile�_parsersr�rrrFr��_ob_short_name�_ob_filenamer��_ob_path�
_target_obr^rMr_r��s^�
�������
	��
zSoSCleaner.obfuscate_filecCs�|jd|jd�|��D]\}z?|�|j�d�d�}|jd||jd�t�|�}tj	�
|j|�|��}|�|�}||ksA||krLt�|�t�
||�Wqtyh}z|�d||f�WYd}~qd}~wwdS)a�Iterate over symlinks in the archive and obfuscate their names.
        The content of the link target will have already been cleaned, and this
        second pass over just the names of the links is to ensure we avoid a
        possible race condition dependent on the order in which the link or the
        target get obfuscated.

        :param archive:     The archive being obfuscated
        :type archive:      ``SoSObfuscationArchive``
        zObfuscating symlink namesr,r�r�zObfuscating symlink %sz"Error obfuscating symlink '%s': %sN)rPr1�get_symlinksrNr�r�rgr=rYr?r@r�rRrZr|)rUr�rZ�_sym�_target�_ob_sym_name�
_ob_targetr�r^r^r_r=(s.

�

�

�����zSoSCleaner.obfuscate_symlinkscCs�|�d|j�t|��dd�D]?}t�|�D]7}tj�||�}|�|j	�d}tj�
|�rN|�|�}||krN|�|�}tj�|j	|�
d�|�}t�||�qqdS)z�For all directories that exist within the archive, obfuscate the
        directory name if it contains sensitive strings found during execution
        z)Obfuscating directory names in archive %sT)�reverser�r�N)rPr1r&�get_directory_listr=�listdirr?r@rNr�r{r��rstripr�rX)rUr��dirpath�_name�_dirname�_arc_dir�_ob_dirname�_ob_arc_dirr^r^r_r<Ks(�

����z$SoSCleaner.obfuscate_directory_namescCsP|jD]"}z|�|�}Wqty%}z
|�d|�WYd}~qd}~ww|S)Nz!Error obfuscating string data: %s)rK�parse_string_for_keysr|rP)rU�string_datar3r�r^r^r_r�`s
��zSoSCleaner.obfuscate_stringcCs�d}|��s
||fS|dur|j}|D]*}z
|�|�\}}||7}Wqty=}z|�d||j�WYd}~qd}~ww||fS)a�Run a line through each of the obfuscation parsers, keeping a
        cumulative total of substitutions done on that particular line.

        Positional arguments:

            :param line str:        The raw line as read from the file being
                                    processed
            :param parsers:         A list of parser objects to obfuscate
                                    with. If None, use all.

        Returns the fully obfuscated line and the number of substitutions made
        rNzfailed to parse line: %s)rOrKrr|rgrL)rUrrrKrFr3�_countr�r^r^r_rVhs��zSoSCleaner.obfuscate_linecCsL|j�d�}|jD]}|�|j�dd����}|�dt|jj	�
���q	dS)zLWrite some cleaner-level, non-report-specific stats to the manifest
        rK� r��entriesN)rJrIrKrLr�rMr3r�r��dataset�keys)rU�	parse_secr3�_secr^r^r_r��s

�z"SoSCleaner.write_stats_to_manifest)NNNFNre)F)NN)*�__name__�
__module__�__qualname__rv�desc�arg_defaultsr5rcrgrPrkrs�classmethodryrBr�r�r�r�rAr�r�r�r�r�r�r�r�r�r
r�rr*r�rr�r=r<r�rVr��
__classcell__r^r^r\r_r'sj&��
F




,^


*
OF#
r)-r�rr9r=r��sos.cleaner.preppersr-rT�concurrent.futuresrr�pwdrr�
sos.componentr�sos.cleaner.parsers.ip_parserr�sos.cleaner.parsers.mac_parserr�#sos.cleaner.parsers.hostname_parserr	�"sos.cleaner.parsers.keyword_parserr
�#sos.cleaner.parsers.username_parserr�sos.cleaner.parsers.ipv6_parserr�sos.cleaner.archives.sosr
rrr�sos.cleaner.archives.genericrr�sos.cleaner.archives.insightsr�
sos.utilitiesrrr�textwraprrr^r^r^r_�<module>s0

@ Telegram