获取 javax.net.ssl.SSLException:在使用 Jsoup 抓取数据时收到致命警报:protocol_version

声明:本页面是StackOverFlow热门问题的中英对照翻译,遵循CC BY-SA 4.0协议,如果您需要使用它,必须同样遵循CC BY-SA许可,注明原文地址和作者信息,同时你必须将它归于原作者(不是我):StackOverFlow 原文地址: http://stackoverflow.com/questions/33595021/
Warning: these are provided under cc-by-sa 4.0 license. You are free to use/share it, But you must attribute it to the original authors (not me): StackOverFlow

提示:将鼠标放在中文语句上可以显示对应的英文。显示中英文
时间:2020-08-11 14:48:26  来源:igfitidea点击:

getting javax.net.ssl.SSLException: Received fatal alert: protocol_version while scraping data using Jsoup

javasslweb-scrapingjsoup

提问by graphics123

I am trying to get data from a site using Jsoup. Link to the site is Click here!

我正在尝试使用 Jsoup 从站点获取数据。该网站的链接是单击此处

Here is my code to fetch the data. `

这是我获取数据的代码。`

    // WARNING: do it only if security isn't important, otherwise you have 
    // to follow this advices: http://stackoverflow.com/a/7745706/1363265
    // Create a trust manager that does not validate certificate chains
    TrustManager[] trustAllCerts = new TrustManager[]{new X509TrustManager(){
        public X509Certificate[] getAcceptedIssuers(){return null;}
        public void checkClientTrusted(X509Certificate[] certs, String authType){}
        public void checkServerTrusted(X509Certificate[] certs, String authType){}
    }};

    // Install the all-trusting trust manager
    try {
        SSLContext sc = SSLContext.getInstance("TLS");
        sc.init(null, trustAllCerts, new SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
    } catch (Exception e) {
        ;
    }`

String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=Starting&SearchStr="+query+"&SearchType=Search"; Connection.Response response = Jsoup.connect(url).timeout(30000) .method(Connection.Method.GET) .userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0")
.execute(); Document document = response.parse();

String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=Starting&SearchStr="+query+"&SearchType=Search"; Connection.Response response = Jsoup.connect(url).timeout(30000) .method(Connection.Method.GET) .userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0")
.execute(); Document document = response.parse();

Please suggest me what is my mistake here.

请建议我在这里我的错误是什么。

采纳答案by Stephan

You want to use Java 8 here since it supports TLSv1.2 by default with additional required cipher suites.

您想在此处使用 Java 8,因为它默认支持 TLSv1.2,并带有其他所需的密码套件。

Why not Java 7?

为什么不是 Java 7?

I tested on my box with Java 7 (1.7.0_45) and got the same error.

我用 Java 7 (1.7.0_45) 在我的盒子上进行了测试,并得到了同样的错误。

I activated the debugging messages and forced TLSv1.2.

我激活了调试消息并强制使用 TLSv1.2。

System.setProperty("javax.net.debug", "all");
System.setProperty("https.protocols", "TLSv1.2");

Then I hit this new error:

然后我遇到了这个新错误:

javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure

Finally, I went to Comodoca's SSL analyzerand see something interesting. According to SSL analyzer, the site you're targeting has only enabled the following cipher suites:

最后,我去了Comodoca 的 SSL 分析器,看到了一些有趣的东西。根据 SSL 分析器,您定位的站点仅启用了以下密码套件:

Cipher Suites Enabled
Name  (ID)                                       Key Size (in bits)
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256  (0xC02F)  128   ECDH 256-bit (P-256) 
TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384  (0xC030)  256   ECDH 256-bit (P-256) 
TLS_DHE_RSA_WITH_AES_128_GCM_SHA256  (0x9E)      128   DH 2048-bit  
TLS_DHE_RSA_WITH_AES_256_GCM_SHA384  (0x9F)      256   DH 2048-bit

(see Full details)

(见完整详情

On my side, I don't have any of the above suites. Check if you have them:

就我而言,我没有上述任何套房。检查您是否拥有它们:

SSLContext sc = SSLContext.getInstance("TLS");
sc.init(null, null, new java.security.SecureRandom());

String[] scs = sc.getSocketFactory().getSupportedCipherSuites();
Arrays.sort(scs);

for(String s : scs) {
   System.out.println(s);
}

See the SSLSocketFactoryExfor enabling the needed cipher suites.

请参阅SSLSocketFactoryEx以启用所需的密码套件。

Why Java 8?

为什么是 Java 8?

On the other hand, I succeed in runnnig the code by moving from Java 7 to Java 8 (1.8.0_20) which support TLS v1.2 by default and provides the needed cipher suites.

另一方面,我通过从 Java 7 迁移到默认支持 TLS v1.2 并提供所需密码套件的 Java 8 (1.8.0_20) 成功地运行了代码。

Here is a trimmed list of supported cipher suites (71 suites in total) for Java 8 (1.8.0_20) on Windows 7.

这是 Windows 7 上 Java 8 (1.8.0_20) 支持的密码套件(总共 71 个套件)的精简列表。

TLS_DHE_RSA_WITH_AES_128_GCM_SHA256
...
TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256

Snippet

片段

try {
    // Create a trust manager that does not validate certificate chains
    TrustManager[] trustAllCerts = new TrustManager[] { new X509TrustManager() {
        public java.security.cert.X509Certificate[] getAcceptedIssuers() {
            return null;
        }

        public void checkClientTrusted(java.security.cert.X509Certificate[] certs, String authType) {
        }

        public void checkServerTrusted(java.security.cert.X509Certificate[] certs, String authType) {
        }
    } };

    // Install the all-trusting trust manager
    SSLContext sc = SSLContext.getInstance("SSL");
    sc.init(null, trustAllCerts, new java.security.SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());

    // Fetch url
    String url = "https://www.sos.nh.gov/corporate/soskb/SearchResults.asp?FormName=CorpNameSearch&Words=All&SearchStr=facebook&SearchType=Search";

    Connection.Response response = Jsoup //
            .connect(url) //
            .timeout(60000) //
            .method(Connection.Method.GET) //
            .userAgent("Mozilla/5.0 (Windows NT 10.0; WOW64; rv:41.0) Gecko/20100101 Firefox/41.0") //
            .execute();

    Document document = response.parse();
    System.out.println(document);
} catch (Exception e) {
    e.printStackTrace();
}

Final thought:

最后的想法:

When it comes to security, ALWAYSuse the latest updated version.

在安全方面,请始终使用最新的更新版本。

回答by dave_thompson_085

(From comment for closure, expanded a little for future finders)

(来自关闭评论,为未来的发现者扩展了一点)

By experiment, that site requires protocol version TLSv1.2 and although Java7 JSSE implements this, client side by default disables1.2 and 1.1. Java8does enable them by default; or in Java7since Jsoup uses HttpsURLConnectionyou can change the enabled versions with system property https.protocols. You need to include at least TLSv1.2and for greatest flexibility should use all currently acceptable protocols https.protocols=TLSv1,TLSv1.1,TLSv1.2.

通过实验,该站点需要协议版本 TLSv1.2,尽管 Java7 JSSE 实现了这一点,但客户端默认禁用1.2 和 1.1。Java8默认启用它们;或者在 Java7 中,因为 Jsoup 使用HttpsURLConnection您可以使用系统属性https.protocols更改启用的版本。您需要至少包括TLSv1.2并且为了最大的灵活性应该使用所有当前可接受的协议https.protocols=TLSv1,TLSv1.1,TLSv1.2

Also, using that all-trusting TrustManagermeans that pretty much any baddie with access to your network can fake this site and expose any sensitive data you send it. It's better to set your local truststore so it accepts the certificate(s) and thus server(s) you need, but not bogus ones.

此外,使用全信任TrustManager意味着几乎任何可以访问您网络的坏人都可以伪造该站点并暴露您发送的任何敏感数据。最好设置您的本地信任库,以便它接受您需要的证书和服务器,而不是伪造的。