Jsoup, Http Error 416, Parsing HTML

I do not know much about jsoup or HTML parsing. I am trying to pull information from whitepages.com try { Document doc = Jsoup.connect('http://www.whitepages.com/phone/1-##

Solution 1:

Okay so Http 416 error occurs when you ask for more data than available, such request is syntactically valid but not satisfiable. For example if you are to request a file of 1K bytes and the actual file on server is less than the requested size, server will issue 416 error, If you request for less bytes than the actual size of server content than you will receive response with http status 206 (Partial Content).

Why is it occurring in your case? Probably and this is my guess not sure, Jsoup is adding range header to your request, see Jsoup.connect(url).maxBodySize() in Jsoup which sets max bytes to read and defaults to 1MB. In your case even if you change this to 200 bytes same error will occur.

Solution : After your Jsoup.connect(url) method add ignoreHttpErrors(true) to ignore such errors, e.g:

        try {
            Document doc = Jsoup.connect("http://www.whitepages.com/phone/1-###-###-####").ignoreHttpErrors(true).get();
            Elements elements = doc.select(".phone-list-data");
            System.out.println(doc.html());
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }