【已恢复正常】【求助】下载脚本失效——存档待验-已解决问题区-AUTOIT CN

holley 发表于 2023-10-7 18:57:57

【已恢复正常】【求助】下载脚本失效——存档待验

本帖最后由 holley 于 2023-10-10 11:04 编辑

原贴来自：怎么正则提取网页内容【已解决】-已解决问题区-AUTOIT CN - Powered by Autoit中文论坛 (autoitx.com)
火绒的真实请求地址：http://www.huorong.cn/versionShow.php

国庆前，正常，今天第一天上班更新发现无法下载了
原脚本：
#include <Inet.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <ProgressConstants.au3>
Opt("GUIOnEventMode", 1)
Local $sSource = _INetGetSource('http://www.huorong.cn/versionShow.php')
Local $sSRERe = StringRegExpReplace($sSource, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1/\/\2/\3')
;~ Local $sSRERe = StringRegExpReplace($sSource, '(?i)^.+All":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1/\/\2/\3')
Local $xName = StringRegExpReplace($sSource, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\3')

$form = GUICreate(StringFormat($xName) & "下载", 600, 120, -1, -1)
Local $xUrl = StringFormat($sSRERe)
Local $nSize = InetGetSize($xUrl)
Local $progressbar1 = GUICtrlCreateProgress(10, 10, 550, 16)
Local $sLabel = GUICtrlCreateLabel("", 570, 12, 30, 16)
GUISetOnEvent($GUI_EVENT_CLOSE, "Quit")
GUISetState()

Local $hDownload = InetGet($xUrl, @ScriptDir & "\\" & $xName, 1, 1) ;不读缓存，后台下载
Do
   Local $aData = InetGetInfo($hDownload) ;
   GUICtrlSetData($progressbar1, Int($aData / $nSize * 100))
   GUICtrlSetData($sLabel, Int($aData / $nSize * 100) & "%")
   Sleep(200)
Until ($aData)
InetClose($hDownload)
WinClose(StringFormat($xName) & "下载")

While 1
   $nMsg = GUIGetMsg()
   Switch $nMsg
            Case $GUI_EVENT_CLOSE
                     Exit
   EndSwitch
WEnd

Func Quit()
   Exit
EndFunc ;==>Quit测试发现 _INetGetSource无法有效获取到网页内容，后改为Http.au3库文件获取，已正常
但同时又发现：InetGet已无法有效下载到$xUrl文件，网页直接下载或下载工具使用正常。（已添加$sUserAgent全局UA）
这属于服务器限制了还是我电脑端的问题？？？我系统win10 专业版。
补充故障现象：
38143
附件已附上Http.au3库文件。
Local $sSource =_HTTP_Get("https://www.huorong.cn/versionShow.php")
翻英文站继续折腾，发现WinHttp库可以下载，但下载下来的东西不对，，，奇怪奇怪真奇怪！
#include <FileConstants.au3>
#include <StringConstants.au3>
#include "WinHttp.au3"

_Example()

Func _Example()
   Local $sDomain = "www.huorong.cn"
   Local $sPage = "/versionShow.php"
   $hOpen = _WinHttpOpen("Mozilla/5.0 (Windows NT 6.1; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0")
   $hConnect = _WinHttpConnect($hOpen, $sDomain)
   $hRequest = _WinHttpOpenRequest($hConnect, "POST", $sPage)
   _WinHttpSendRequest($hRequest, "Content-Type: application/x-www-form-urlencoded")
   _WinHttpReceiveResponse($hRequest)
   Dim $sReturned
   If _WinHttpQueryDataAvailable($hRequest) Then ; 如果有数据
            Do
                     $sReturned &= _WinHttpReadData($hRequest)
            Until @error
   EndIf ; 关闭句柄

   _WinHttpCloseHandle($hRequest)
   _WinHttpCloseHandle($hConnect)
   _WinHttpCloseHandle($hOpen)
   Local $sSRERe = StringRegExpReplace($sReturned, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1/\/\2/\3')
   Local $sSRERe1 = StringRegExpReplace($sReturned, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1')
   Local $sSRERe2 = StringRegExpReplace($sReturned, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\2')
   Local $xName = StringRegExpReplace($sReturned, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\3')
   ; "YOUR_SOURCE_URL"
   Local $EXAMPLE_URL = StringFormat($sSRERe)

   ; "YOUR_DESTINATION"
   Local $LOCAL_FILE = @ScriptDir & "\" & $xName

;~ _WinHTTP_GetAndSaveFileToDisc_Ex($EXAMPLE_URL, $LOCAL_FILE, 0)
   _WinHTTP_GetAndSaveFileToDisc_Ex($EXAMPLE_URL, $LOCAL_FILE, 1)
;~ _WinHTTP_GetAndSaveFileToDisc_Ex($EXAMPLE_URL, $LOCAL_FILE, 2)
EndFunc ;==>_Example

Func _WinHTTP_GetAndSaveFileToDisc_Ex($EXAMPLE_URL, $LOCAL_FILE, $iMode = 1)
   Local $hTimer = TimerInit()
   Local $hOpen = _WinHttpOpen() ; Initialize and get session handle
   Local $s_URL_Domain = StringRegExpReplace($EXAMPLE_URL, '(?i)(.*?\..*?)(\/.+)', '$1')
   Local $hConnect = _WinHttpConnect($hOpen, $s_URL_Domain, $INTERNET_DEFAULT_HTTPS_PORT) ; Get connection handle

   Local $s_URL_File = StringReplace($EXAMPLE_URL, $s_URL_Domain, '')
   Local $hRequest = _WinHttpOpenRequest($hConnect, Default, $s_URL_File, Default, Default, Default, $WINHTTP_FLAG_SECURE)
   If @error Then ConsoleWrite("! ---> @error=" & @error & "@extended=" & @extended & _
                     " : _WinHttpOpenRequest" & @CRLF)

   _WinHttpSendRequest($hRequest) ; Send request

   _WinHttpReceiveResponse($hRequest) ; Wait for the response

   ProgressOn("Downloading", "In __WinHTTP_DownloadProgress...")
   __WinHTTP_DownloadProgress(_WinHttpQueryHeaders($hRequest, $WINHTTP_QUERY_CONTENT_LENGTH))

   Local $sData, $sChunk
   If _WinHttpQueryDataAvailable($hRequest) Then ; Check if there is data available...
            While 1
                     $sChunk = _WinHttpReadData_Ex($hRequest, $iMode, Default, Default, __WinHTTP_DownloadProgress)
                     If @error Then ExitLoop
                     $sData &= $sChunk
;~                      Sleep(50)
            WEnd
   Else
            MsgBox(48, "Error", "Site is experiencing problems (or you).")
   EndIf

   Sleep(100)
   ProgressOff()

   ; Close handles
   _WinHttpCloseHandle($hRequest)
   _WinHttpCloseHandle($hConnect)
   _WinHttpCloseHandle($hOpen)

   If Not @Compiled Then ConsoleWrite("! IFNC: BinaryLen($sData) = " & BinaryLen($sData) & @CRLF)
   Local $hFile = FileOpen($LOCAL_FILE, $FO_OVERWRITE + $FO_CREATEPATH + $FO_BINARY + $FO_UTF8)

   FileWrite($hFile, $sData)

   FileClose($hFile)
   If Not @Compiled Then ConsoleWrite("> IFNC: " & TimerDiff($hTimer) / 1000 & @CRLF)
EndFunc ;==>_WinHTTP_GetAndSaveFileToDisc_Ex

Func __WinHTTP_DownloadProgress($iSizeAll, $iSizeChunk = 0)
   Local Static $iMax, $iCurrentSize
   If $iSizeAll Then $iMax = $iSizeAll
   $iCurrentSize += $iSizeChunk
   If Not @Compiled Then ConsoleWrite("! IFNC: $iCurrentSize = " & $iCurrentSize & @CRLF)
   Local Static $iPercent_static = 0

   Local $iPercent = Round($iCurrentSize / $iMax * 100, 0)
   If $iPercent_static <> $iPercent Then
            $iPercent_static = $iPercent
            ProgressSet($iPercent, $iPercent & " %")
   EndIf
EndFunc ;==>__WinHTTP_DownloadProgress

Func _WinHttpReadData_Ex($hRequest, $iMode = Default, $iNumberOfBytesToRead = Default, $pBuffer = Default, $vFunc = Default)
   __WinHttpDefault($iMode, 0)
;~       __WinHttpDefault($iNumberOfBytesToRead, 8192)
   __WinHttpDefault($iNumberOfBytesToRead, 1024)
   __WinHttpDefault($vFunc, 0)
   Local $tBuffer, $vOutOnError = ""
   If $iMode = 2 Then $vOutOnError = Binary($vOutOnError)
   Switch $iMode
            Case 1, 2
                     If $pBuffer And $pBuffer <> Default Then
                           $tBuffer = DllStructCreate("byte[" & $iNumberOfBytesToRead & "]", $pBuffer)
                     Else
                           $tBuffer = DllStructCreate("byte[" & $iNumberOfBytesToRead & "]")
                     EndIf
            Case Else
                     $iMode = 0
                     If $pBuffer And $pBuffer <> Default Then
                           $tBuffer = DllStructCreate("char[" & $iNumberOfBytesToRead & "]", $pBuffer)
                     Else
                           $tBuffer = DllStructCreate("char[" & $iNumberOfBytesToRead & "]")
                     EndIf
   EndSwitch
   Local $sReadType = "dword*"
   If BitAND(_WinHttpQueryOption(_WinHttpQueryOption(_WinHttpQueryOption($hRequest, $WINHTTP_OPTION_PARENT_HANDLE), $WINHTTP_OPTION_PARENT_HANDLE), $WINHTTP_OPTION_CONTEXT_VALUE), $WINHTTP_FLAG_ASYNC) Then $sReadType = "ptr"
   Local $aCall = DllCall($hWINHTTPDLL__WINHTTP, "bool", "WinHttpReadData", _
                     "handle", $hRequest, _
                     "struct*", $tBuffer, _
                     "dword", $iNumberOfBytesToRead, _
                     $sReadType, 0)
   If @error Or Not $aCall Then Return SetError(1, 0, "")
   If Not $aCall Then Return SetError(-1, 0, $vOutOnError)
   If IsFunc($vFunc) Then $vFunc(0, $aCall)
   Local $dBinary = DllStructGetData($tBuffer, 1)
   If $aCall < $iNumberOfBytesToRead Then
            Switch $iMode
                     Case 0
                           Return SetExtended($aCall, StringLeft($dBinary, $aCall))
                     Case 1
                           Return SetExtended($aCall, BinaryToString(BinaryMid($dBinary, 1, $aCall), $SB_ANSI))
                     Case 2
                           Return SetExtended($aCall, BinaryMid($dBinary, 1, $aCall))
            EndSwitch
   Else
            Switch $iMode
                     Case 0, 2
                           Return SetExtended($aCall, $dBinary)
                     Case 1
                           Return SetExtended($aCall, BinaryToString($dBinary, $SB_ANSI))
            EndSwitch
   EndIf
EndFunc ;==>_WinHttpReadData_Ex控制台可以看到，当前大小读取是正常的，但最后保存的时候感觉像截断了保存下来的。

gyp2000 发表于 2023-10-7 19:11:18

下载前，设置下AU3的浏览器UA标识试一试。这样可以模拟浏览器下载。无法下载可能是对方服务器做了反爬虫设置。

gyp2000 发表于 2023-10-7 20:54:08

本帖最后由 gyp2000 于 2023-10-7 20:56 编辑

测试了，函数下载正常，是你代码中路径写错了，缺少斜杠。Local $hDownload = InetGet($xUrl, @ScriptDir & "\" & $xName, 1, 1) ;不读缓存，后台下载

holley 发表于 2023-10-8 09:01:09

gyp2000 发表于 2023-10-7 20:54
测试了，函数下载正常，是你代码中路径写错了，缺少斜杠。Local $hDownload = InetGet($xUrl, @ScriptDir & ...

感谢回复！有斜杠应该是粘贴出来自动丢失了

afan 发表于 2023-10-8 14:48:52

win10测试了一下，用1#的原代码正常下载

gyp2000 发表于 2023-10-9 19:26:49

可以试试这样：
修改了下下载完成检测的循环部分。

#include <Inet.au3>
#include <GUIConstantsEx.au3>
#include <WindowsConstants.au3>
#include <ProgressConstants.au3>
Opt("GUIOnEventMode", 1)
Local $sSource = _INetGetSource('http://www.huorong.cn/versionShow.php')
Local $sSRERe = StringRegExpReplace($sSource, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1/\/\2/\3')
;~ Local $sSRERe = StringRegExpReplace($sSource, '(?i)^.+All":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\1/\/\2/\3')
Local $xName = StringRegExpReplace($sSource, '^.+urlAll":"(h.+?:)[\\/]+([\w.]+)[\\/]+([^"]+).+$', '\3')

$form = GUICreate(StringFormat($xName) & "下载", 600, 120, -1, -1)
Local $xUrl = StringFormat($sSRERe)
Local $nSize = InetGetSize($xUrl)
Local $progressbar1 = GUICtrlCreateProgress(10, 10, 550, 16)
Local $sLabel = GUICtrlCreateLabel("", 570, 12, 30, 16)
GUISetOnEvent($GUI_EVENT_CLOSE, "Quit")
GUISetState()

Local $hDownload = InetGet($xUrl, @ScriptDir & "\" & $xName, 1, 1) ;不读缓存，后台下载
Local $aData
Do
$aData = InetGetInfo($hDownload) ;
GUICtrlSetData($progressbar1, Int($aData / $nSize * 100))
GUICtrlSetData($sLabel, Int($aData / $nSize * 100) & "%")
Sleep(5)
Until InetGetInfo($hDownload, 2)
GUICtrlSetData($progressbar1, 100)
GUICtrlSetData($sLabel, "100%")
Sleep(1000)
WinClose(StringFormat($xName) & "下载")

$aData = InetGetInfo($hDownload)
If @error Then
FileDelete(@ScriptDir & "\" & $xName)
EndIf

InetClose($hDownload)

While 1
$nMsg = GUIGetMsg()
Switch $nMsg
Case $GUI_EVENT_CLOSE
Exit
EndSwitch
WEnd

Func Quit()
Exit
EndFunc ;==>Quit

页: [1]

AUTOIT CN's Archiver

【已恢复正常】【求助】下载脚本失效——存档待验