java - Dealing with pagination in web pages while using jsoup -


I am using jsoup to crawl through webpages of a particular website. In fact, I am trying to remove all those who have a PDF link. I am able to get all the links to a particular page but 10 uses the web page logic JavaScript _doPostBack () function to navigate to other pages are pages I could make it to get by Jesoupi.

This is how I'm trying it now 

  Document document = Jsoup.connect ( "a website name") Kdeta ( "__EVENTARGUMENT", __EVENTARGUMENT). Data ( "__EventTarGET", __EVENTTARGET) Kdeta ( "__ Events", __EVENTVALIDATION) Kdeta ( "__ Vivekkrta", __VIEWSTATEGENERATOR) .cookie ( "ASP.NET_SessionId", session IDs) KfollowRirectirects (true). Timeout (0) Kyujragent ( "Mozilla / 5.0 (Windows; U; Windows NT 5.1; en-US; rv Lk8klk6) Gecko / 20070725 Firefox / 2.0.0.6") Kpost ();

But I'm getting a false URL output. I have defined all variables before sending them.

When I pressed the issue like this, I am here to solve I have them:

  • Load pages in a browser
  • Spy HTTP messages to be e-mailed between pages and browsers going through the pages (Fiddler, Firebug, Developer Console / Toolbar). ..)
  • Identify each single byte browser and server exchanges (headers, cookies, etc.)
  • Once all the single bytes Once identified, try to go through the pages (header, cookies, user-agent, etc.)
  • Once you are able to go through the pages with hurl.it, Jesop Instructs to do this

Comments

Post a Comment

Popular posts from this blog

python - Overriding the save method in Django ModelForm -

I'm having trouble overriding a ModelForm save method. I get this error I'm: Exception type: TypeErier Exception Value: Save () found an unexpected keyword argument 'committed' My intentions include a form 3 fields Submit several values ​​for, then create an object for each combination of those areas, and to save each object file models.py class collarsult type (models.Model): id = model. AutoField (db_column = 'icontact_result_code_type_id', primary_key = true) callResult = models.ForeignKey ( 'CallResult', db_column = 'icontact_result_code_id') campaign = models.ForeignKey ( 'campaign', db_column = 'icampaign_id') CALLTYPE = models.ForeignKey ( 'CALLTYPE', db_column = 'icall_type_id') agent = models.BooleanField (db_column = 'bagent', default = true) teamLeader = models.BooleanField (db_column = 'bTeamLeader', default = true) active = models.BooleanField (db_column = Django.form...
Read more

html - CSS autoheight, but fit content to height of div -

I have a div in which there are three children- div in the left and middle div: float: left while true: float: true because it is my layout Actually will spoil, I've used ClearFix Hack: .cf {zoom: 1; }. Cf: First,. Cf: {content: ""; Display: Table; }. Cf: After {clear: both; } It still works, but I want to be the right div an indicator. So it should fill 100% of the parent unit of height. How can I complete this? PS is the full code: & Lt; / Div & gt; & Lt; Div class = "mobile_content" & gt; & Lt; H5 & gt; {{Data.titel}} & lt; / H5> & Lt; Table & gt; & Lt; TR & gt; & Lt; Th & gt; Komponist: & lt; / Th & gt; & Lt; TD & gt; {{Data.komponist}} & lt; / TD & gt; & Lt; / TR & gt; & Lt; TR & gt; & Lt; Th & gt; Instrument: & lt; / Th & gt; & Lt; TD & gt; {{Data.instrumente}} & lt; / TD & gt; & Lt; / TR & gt;...
Read more

qt - How to prevent QAudioInput from automatically boosting the master volume to 100%? -

I have been trying to use Qt5 multimedia to record audio with QAudioInput, however, when I see If QAudioInput is started, it increases the master volume of my sound device to 100%. How can I prevent QAudioInput from changing the master volume? My current development platform is Linux with PalsAdio (with flat audio disabled). How can I use QAudioInput : QAudioDeviceInfo device_info = QAudioDeviceInfo :: defaultInputDevice (); QAudio format format; Format.setSampleRate (44100); Format.setChannelCount (1); Format.setSampleSize (16); Format.setCodec ("Audio / PCM"); Format.setSampleType (QAudioFormat :: SignedInt); Format.setByteOrder (QAudioFormat :: LittleEndian); Std :: cout & lt; & Lt; Device_info.deviceName (). ToUtf8 (). ConstData () & lt; & Lt; Std :: endl; QAudioInput * default_device = new QAudioInput (device_info, format); QIODevice * default_io_device = default_device-> Start (); a QAudioInput.setVolume () method is not you I ...
Read more