Today we talk about SuperWebClient A few simple topics in using components

1: UserAgent
2: Cookies
3: POST Sign in

This is the client identification information , This information is used to identify that you are accessing WEB The type of client for the service , Now we use the bag grabbing tool Fiddler, To get a few typical browsers
What's this information like . Fiddler It's a .NET The open source HTTP/HTTPS Protocol analysis software , Later, after the acquisition, there was no open source, like .
Fiddler Download address, After operation, it is shown as follows

First let's open up IE browser - Enter the blog Garden website , You will see Fiddler Caught a lot of packets , We can select the one on the front page to view the specific http Request information

We can know IE Browser's userAgent The information is as follows
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)
( On the machine IE The version is different from the software installed ,USER-AGENT There will be some differences )
We're looking at Google based CHROME The browser is similar to the following
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.48 Safari/537.36
And a lot of cheetahs , firefox , Roam and others have their own additional information in User-Agent in , What kind of use might this have .
generally speaking ,WEB The server doesn't value this information , But some will also check , And give appropriate output feedback according to different clients , The other is the client JS The script will be based on
Check and adjust this information for compatibility .
         For crawlers like Baidu , Google and others have their own unique userAgent Information ,SuperWebClient The default is IE Browser's , Of course , You can set up
Simulate any information , Like the following SuperWebClient The application code example is as follows
Notice the red area of the code , It was originally discussed as a later topic , Now it's to show you how to use FIDDLER Grab the SuperWebClient The request package will be marked in advance
The default is simulated IE Browser's , Now I'm going to modify this User-Agent The message is “i-love-you”

  1. // First build a hi object - That is, the input object of the simulation access
  2. HttpInput hi = new HttpInput();
  3. // You need to initialize this hi, Set whether to enable HTTP 1.1, Next is the connection and transmission timeout settings ( second )
  4. HttpManager.Instance.InitWebClient(hi, true, 60, 60);

Copy code

                  // Agent settings - Otherwise FIDDLER It's impossible to grasp
                            hi.EnableProxy = true;
                            hi.ProxyIP = "";
                            hi.ProxyPort = 8888;

hi.UserAgent = "i-love-you";            
                         // The next step is to set up access URL, And other things like UserAgent,Cookies,Proxy.... And so on, all kinds of feature settings
                         // commonly , Let's set up Url Access path , Everything else is set when necessary

    1. hi.Url = "";
    2. // There are input objects , There is a corresponding output object - Be careful , This method is blocked , Only when the results come back
    3. // therefore , You can put the code into a separate thread or thread pool for data collection
    4. HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
    5. if (ho.IsOK)
    6. {
    7. // If HO The logo of IsOK establish , The page access is successful , Otherwise, the page access fails
    8. // Encounter failure - You can do it yourself , For example, re request or write LOG
    9. richTextBox1.Text = ho.TxtData;
    10. }
    11. else
    12. {
    13. richTextBox1.Text = " Page access error ";
    14. }
    15. // Finally, destroy the object
    16. hi.Dispose();

You can pretend to be whatever you need , For example, wechat , The unique information of Momo .
SuperWebClient Default processing WEB From the server COOKIE, You don't have to think about it yourself , That's from the server COOKIE How to read information

HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
if (ho.IsOK)
richTextBox2.Text = ho.Cookies; // If HO The logo of IsOK establish , The page access is successful , Otherwise, the page access fails
// Encounter failure - You can do it yourself , For example, re request or write LOG
richTextBox1.Text = ho.TxtData;

In the output object ho.Cookies In the attribute .

For example, visit C# Ant nest

To get the cookies yes
WEB The server will send different cookies, SuperWebClient Output object of httpoutput It's just sent by the server at the time of the request COOKIES Information , It doesn't accumulate , although SuperWebClient Internally, all the messages sent from the server will be accumulated COOKIE And send it back when it is submitted to the server , But it's not available to read all the cumulative COOKIE Methods , You can collect feedback on each request by yourself httpoutput Medium COOKIES Information , generally speaking , I don't pay much attention to this thing , Unless you want to save COOKIES Do it another way .

3:POST Sign in
         In general , If no accident ,WEB Many operation requests of services are based on POST Agreed ,AJAX It's also , Very little is based on GET Agreed , Because the request string is length limited
A lot of parameters need to be passed through POST To send the , Next , Let's see. SuperWebClient How to use POST How to log on to a website , Before the show , We need to analyze the website login
Request agreement for , And then use SuperWebClient To simulate .
Let's explain it with the process of logging into the blog park , First of all, we use Fiddler Capture the login protocol of blog park , Be careful , If you can't get the login protocol of the blog Park , See below 2 individual FIDDLER Two of the most common problems in , Because blog Park login is adopted HTTPS, So it needs to be turned on FIDDLER Corresponding HTTPS Decoding options .

If you come across something you can grab , But it's hard to decode , That's your certificate problem , Certificate problem - You can refer to  here - Rebuild the certificate - The problem will be solved

Here is the login protocol of the blog park

x-requested-with: XMLHttpRequest
Accept-Language: zh-cn
Accept: application/json, text/javascript, */*; q=0.01
verificationtoken: _Q9B84azUZnUTIffmygxm8pONeAnXanp7gw7sqep3b3Lqk-opvj5wPgeAN6SM6lN1qFi6y8-36GAyjMDYYBuf_cy6iE1:fmSuVFj_iIGBnMV7N3TydHUuc8ORlIkNw7eSCsGqwV2BX7lM64bNoMV0X3S-t2-rlDcBpfuYnK3_In_fMNTFaPhE3Cw1
Content-Type: application/json; charset=utf-8
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)
Content-Length: 386
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: Hm_lvt_cc17b07fc9529e3d80b4482c9ce09086=1482645471; _ga=GA1.2.806287383.1482645352; __gads=ID=cde93307386572b7:T=1488262791:S=ALNI_MbB1oiDaPhuYt-REwWv3uYFQG8-AA; AspxAutoDetectCookieSupport=1; SERVERID=227b087667da6f8e99a1165002db93f6|1488285061|1488285061 {"input1":"a9Q+6M4EwC0HlHafyL8GYvd5zcFP+tUMV+F2P+aF0FdzrtY9nqPdkjCDctPQlY37EI/VzpLc9z+SAKXaWpWa0yDUM3USN+b7W+m6t7p8qRS+nuHu1G6wJ34IyuAruQ5gE3+r30PKy65q9dD0F5ruOSZizIYgCvm2/WrQfQsdYiQ=","input2":"fGOIEUEG5HP6tt3c/q9z0DuLyc9qeq0GiJQyTGNhTL/sCriRomR2G8gtn+AurDFzH/dDzMOYrBfEJ97n454gg8Ppd5cRe3qaNBTPebCJrIK3chv/Mupe+liFvEu6MwOgPqFBRm6GQCSZBHaVfRpHBKYrQkEaWRZ6ItSjKCBKpQc=","remember":false}

The account number and password are encrypted , As for the encryption process , You need to carefully analyze the blog park's JS Code , Here we are just to illustrate how to use it SuperWebClient Conduct
POST Form login , So for the account number , The cryptanalysis of the password is omitted , If there's a lot of free time in the back , I will add an article to analyze the account number of the blog park , Password encryption , Mainly related to
JS Debugging and analysis of code .

So use SuperWebClient How to log in to the blog Park . It's simple .

HttpInput hi = new HttpInput();
HttpManager.Instance.InitWebClient(hi, true, 60, 60);
// Set up blog Park login here URL
hi.Url = "";
hi.Refer = "";
// We forge down the road , Prove where we're from , According to the agreement
// POST data hi.PostData = "input1=a9Q+6M4EwC0HlHafyL8GYvd5zcFP+tUMV+F2P+aF0FdzrtY9nqPdkjCDctPQlY37EI/Vzp"
+"&remember=false"; HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
if (ho.IsOK)
{ richTextBox1.Text = ho.TxtData;
richTextBox1.Text = " Page access error ";
} // Finally, destroy the object

hi.PostData=" Set up POST The form data "       The format is :   key1=value1&key2=value2..... In turn

Today's topic is over , With some of today's stuff , You've been able to handle most of the mock requests ,SuperWebClient Most of the features of are also around the present GET,POST service .

SuperWebClient - One is based on CURL Of .NET HTTP/HTTPS Analog God component (2) More articles about

  1. SuperWebClient - One is based on CURL Of .NET HTTP/HTTPS Analog God component (1)

    We all know , Whether you're a reptile or not , It's a collection tool , Their HTTP/HTTPS Mock access is always a fundamental issue , I guess there are a lot of people like me , be that as it may , That kind of built-in or third-party class library uses a lot of , But there are always some unsatisfactory problems , also ...

  2. One is based on nodejs, Support http/https The broker in (MITM) agent , Easy for penetration testing and development debugging .

    Source code address : node-mitmproxy node-mitmproxy It's based on nodejs, Support http/h ...

  3. Hand in hand write one based on Spring Boot Parameter verification component under the framework (JSR-303)

    Preface In the process of the research and development of the new open platform , Because different interfaces need to verify different input parameters , This involves the verification and encapsulation of general parameters , If it's not packaged , Then the verification code will be written in different styles . The verification tool class is inconsistent . High maintenance risk and other factors ...

  4. Yoshino: One is based on React Can be customized PC Component library

    Github: Docs: Cli-Tool: https://git ...

  5. 【 original 】 It's based on .NET Of Redis Cluster Cluster access components

    Hello Hello everyone , I am a TANZAME, We meet again . Today we're going to talk about how to make one  Redis Cluster Cluster clients , There are dry goods by hand , Your fine products . As the business grows , Online environment QPS bulge , Naturally, the current single ...

  6. Simulated Login artifact PHP be based on cURL Implementation of automatic simulation Login Class

    One . conceived from Firefox Browser copy cURL command ( Initial page . Submit . After submission ) Automatic analysis curl Form a simulated login code Default parameters :ssl/302/gzip Two . Realization Interface ( One ) according to curl Information execution and parsing results p ...

  7. ordinary php be based on curl The reverse agent for

    cause : The manager : You need to implement a reverse proxy ? I :   Simple ,nginx Set it up every minute . The manager : Um. ? No, nginx? I : nodejs That's fine , There are examples on the Internet that can be done in minutes . The manager : Um. ? Only virtual hosts , Can only upload php Program ? ...

  8. SDKMAN A command line interface based SDK User environment manager

    1. background Have used Python Friends of development , It should be understood that Python2 and Python3 Grammatical differences , Sometimes you download code from the Internet based on different interpreters , To switch versions back and forth , It's not very convenient to use , Sometimes it's even troublesome . So someone invented ...

  9. psutil One is based on python Cross platform system information tracking module

    Thanks to this module , Here I recommend a hand . psutil It's based on python Cross platform system information monitoring module . stay pytho ...

Random recommendation

  1. ik Extended support Solr To configure

    Expand ik Native code :public class IKAnalyzerTokenizerFactory extends TokenizerFactory{ private boolean useSmart; ...

  2. About PHP Of curl Open the question

    I'm doing a Sina login function today , Sina offers PHP SDK You need to use curl_init() function , It can't be found during debugging curl_init() Methods .  We need to open it after searching curl Of php Expand , that curl And what is it ...

  3. Javascript The operation table is interlaced

    <!DOCTYPE html> <html xmlns=""> <head> <m ...

  4. python+robot framework Interface automation testing

    python+requests The request to implement the interface has been described in the previous article , You can also visit if you don't understand or have questions python+request Interface automation framework At present, we need to consider how to achieve keyword driven interface automation output , By keyword ...

  5. thorough Web Request process

    B/S Network architecture Benefits : 1. The client uses the same browser . -- The interactive nature of the browser makes it very easy to use 2. The server is based on a unified http.  -- simplify . Standardize the development mode , Greatly reduce development costs . Such as tomcat ngi ...

  6. npm Release vue Component flow

    Initialize project vue init webpack-simple XXX Define the components Release configuration 1.package.json 2.webpack.config.js( The comment section is the original configuration ) Release 1. Sign in 2. Release n ...

  7. https Self signed SSL certificate

    Introduce TLS Or transport layer security , And its predecessor SSL( Represents the secure socket layer ) Is used to wrap normal traffic in a protected encryption wrapper Web agreement . Using this technology , The server can safely send traffic between the server and the client , Without being intercepted by outside parties ...

  8. C# Use CefSharp Open source library development Chrome browser

    One . Introduce          I haven't touched this before , But it's useful in corporate projects , So I'll look at it in passing . I just did it today WinForm Test of , Try it when you have time WPF How is it realized . It was a mess at first , It's a bit numb ...

  9. About CoreData Usage of

    Some colleagues feel that CoreData It's a person who doesn't understand , Mysterious things that can't be understood , Actually ios Local data storage is a sqlite database , A simple database , And this CoreData Does it support all stored data , Obviously not , Standing in my corner ...

  10. upsource On

    stay JetBrains On our website , notice codereview Tools for upsource ,   Official English documents Let's take a look at the bloggers in the blog park ...