ï»¿
Your IP : 172.28.240.42

Current Path : /usr/lib/python2.7/dist-packages/landscape/lib/
Current File : //usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyc
�
���Tc@sddlZddlZddlmZddlmZddlmZddlmZde	fd��YZ
de
fd	��YZd
e
fd��YZe
diddd
de
edd�
Zd�Zddd�Zdd�Zdd�Zd�Zedkreejd�ndS(i����N(tOptionParser(tStringIO(t
deferToThread(tDeferredListt
FetchErrorcBseZRS((t__name__t
__module__(((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyRst
HTTPCodeErrorcBs#eZd�Zd�Zd�ZRS(cCs||_||_dS(N(t	http_codetbody(tselfRR	((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyt__init__s	cCsd|jS(NsServer returned HTTP code %d(R(R
((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyt__str__scCsd|jS(Ns<HTTPCodeError http_code=%d>(R(R
((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyt__repr__s(RRRRR
(((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyRs		tPyCurlErrorcBs2eZd�Zd�Zd�Zed��ZRS(cCs||_||_dS(N(t
error_codet_message(R
Rtmessage((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyRs	cCsd|j|jfS(NsError %d: %s(RR(R
((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyR"scCsd|j|jfS(Ns<PyCurlError args=(%d, '%s')>(RR(R
((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyR
%s	cCs|jS(N(R(R
((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyR)s(RRRRR
tpropertyR(((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyRs			tiiXcCs�ddl}t|�}t�}
|dkr<|j�}n|j|jt|��|r�|j|jt�|r�|j|j	t
|��|j|j|j�q�n|r�|j
d�r�|j|j|�n|r|j|jgt|j��D]}d|^q��n|r/|j|jt�n|	rK|j|jd�n|
dk	rm|j|j|
�n|j|jd�|j|j|�|j|jd�|j|j|�|j|jd�|j|j|
j�|j|jd�|j|jd�y|j�Wn3|j k
rK}t!|j"d|j"d��nX|
j#�}|j$|j%�}|d	kr�t&||��n|S(
sRetrieve a URL and return the content.

    @param url: The url to be fetched.
    @param post: If true, the POST method will be used (defaults to GET).
    @param data: Data to be sent to the server as the POST content.
    @param headers: Dictionary of header => value entries to be used on the
        request.
    @param curl: A pycurl.Curl instance to use. If not provided, one will be
        created.
    @param cainfo: Path to the file with CA certificates.
    @param insecure: If true, perform curl using insecure option which will
        not attempt to verify authenticity of the peer's certificate. (Used
        during autodiscovery)
    @param follow: If True, follow HTTP redirects (default True).
    @param user_agent: The user-agent to set in the request.
    i����Nshttps:s%s: %siiisgzip,deflatei�('tpycurlRtNonetCurltsetopttURLtstrtPOSTtTruet
POSTFIELDSIZEtlentREADFUNCTIONtreadt
startswithtCAINFOt
HTTPHEADERtsortedt	iteritemstSSL_VERIFYPEERtFalsetFOLLOWLOCATIONt	USERAGENTt	MAXREDIRStCONNECTTIMEOUTtLOW_SPEED_LIMITtLOW_SPEED_TIMEtNOSIGNALt
WRITEFUNCTIONtwritetDNS_CACHE_TIMEOUTtENCODINGtperformterrorRtargstgetvaluetgetinfot	HTTP_CODER(turltposttdatatheaderstcainfotcurltconnect_timeoutt
total_timeouttinsecuretfollowt
user_agentRtoutputtinputtpairteR	R((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pytfetch.sN	-!cOstt||�S(s]Retrieve a URL asynchronously.

    @return: A C{Deferred} resulting in the URL content.
    (RRG(R4tkwargs((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pytfetch_asyncxscKs{g}x\|D]T}t||�}|r;|j||�n|rT|j||�n|j|�q
Wt|dtdt�S(s9
    Retrieve a list of URLs asynchronously.

    @param callback: Optionally, a function that will be fired one time for
        each successful URL, and will be passed its content and the URL itself.
    @param errback: Optionally, a function that will be fired one time for each
        failing URL, and will be passed the failure and the URL itself.
    @return: A C{DeferredList} whose callback chain will be fired as soon as
        all downloads have terminated. If an error occurs, the errback chain
        of the C{DeferredList} will be fired immediatly.
    tfireOnOneErrbackt
consumeErrors(RItaddCallbackt
addErrbacktappendRR(turlstcallbackterrbackRHtresultsR8tresult((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pytfetch_many_async�s
cCsD|jd�jd�d}|dk	r@tjj||�}n|S(s�Return the last component of the given C{url}.

    @param url: The URL to get the filename from.
    @param directory: Optionally a path to prepend to the returned filename.

    @note: Any trailing slash in the C{url} will be removed
    t/i����N(trstriptsplitRtostpathtjoin(R8t	directorytfilename((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyturl_to_filename�scs7�fd�}�fd�}t|d|d||�S(sJ
    Retrieve a list of URLs and save their content as files in a directory.

    @param urls: The list URLs to fetch.
    @param directory: The directory to save the files to, the name of the file
        will equal the last fragment of the URL.
    @param logger: Optional function to be used to log errors for failed URLs.
    cs<t|d��}t|d�}|j|�|j�dS(NR[tw(R]topenR/tclose(R:R8R\tfd(R[(s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyR/�s
cs*�r&�d|t|j�f�n|S(Ns Couldn't fetch file from %s (%s)(Rtvalue(tfailureR8(tlogger(s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyt	log_error�sRPRQ(RT(ROR[RdRHR/Re((R[Rds7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pytfetch_to_files�s
cCs~t�}|jddd�|jddd�|jd�|j|�\}\}t|d|jd	|jd
|j�GHdS(Ns--posttactiont
store_trues--datatdefaultRs--cainfoR9R:R<(Rt
add_optiont
parse_argsRGR9R:R<(R4tparsertoptionsR8((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyttest�s	
t__main__i(RXtsystoptparseRRttwisted.internet.threadsRttwisted.internet.deferRt	ExceptionRRRR&RRRGRIRTR]RfRnRtargv(((s7/usr/lib/python2.7/dist-packages/landscape/lib/fetch.pyt<module>s$
H