Today we talk about SuperWebClient A few simple topics in using components

1: UserAgent
2: Cookies
3: POST Sign in

1:UserAgent
This is the client identification information , This information is used to identify that you are accessing WEB The type of client for the service , Now we use the bag grabbing tool Fiddler, To get a few typical browsers
What's this information like . Fiddler It's a .NET The open source HTTP/HTTPS Protocol analysis software , Later, after the acquisition, there was no open source, like .
Fiddler Download address  http://www.telerik.com/fiddler, After operation, it is shown as follows

First let's open up IE browser - Enter the blog Garden website , You will see Fiddler Caught a lot of packets , We can select the one on the front page to view the specific http Request information

We can know IE Browser's userAgent The information is as follows
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)
( On the machine IE The version is different from the software installed ,USER-AGENT There will be some differences )
We're looking at Google based CHROME The browser is similar to the following
User-Agent: Mozilla/5.0 (Windows NT 6.3; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.48 Safari/537.36
And a lot of cheetahs , firefox , Roam and others have their own additional information in User-Agent in , What kind of use might this have .
generally speaking ,WEB The server doesn't value this information , But some will also check , And give appropriate output feedback according to different clients , The other is the client JS The script will be based on
Check and adjust this information for compatibility .
         For crawlers like Baidu , Google and others have their own unique userAgent Information ,SuperWebClient The default is IE Browser's , Of course , You can set up
Simulate any information , Like the following SuperWebClient The application code example is as follows
Notice the red area of the code , It was originally discussed as a later topic , Now it's to show you how to use FIDDLER Grab the SuperWebClient The request package will be marked in advance
The default is simulated IE Browser's , Now I'm going to modify this User-Agent The message is “i-love-you”

  1. // First build a hi object - That is, the input object of the simulation access
  2. HttpInput hi = new HttpInput();
  3. // You need to initialize this hi, Set whether to enable HTTP 1.1, Next is the connection and transmission timeout settings ( second )
  4. HttpManager.Instance.InitWebClient(hi, true, 60, 60);

Copy code

                  // Agent settings - Otherwise FIDDLER It's impossible to grasp
                            hi.EnableProxy = true;
                            hi.ProxyIP = "127.0.0.1";
                            hi.ProxyPort = 8888;

hi.UserAgent = "i-love-you";            
                         // The next step is to set up access URL, And other things like UserAgent,Cookies,Proxy.... And so on, all kinds of feature settings
                         // commonly , Let's set up Url Access path , Everything else is set when necessary

    1. hi.Url = "http://www.cnblogs.com/";
    2. // There are input objects , There is a corresponding output object - Be careful , This method is blocked , Only when the results come back
    3. // therefore , You can put the code into a separate thread or thread pool for data collection
    4. HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
    5. if (ho.IsOK)
    6. {
    7. // If HO The logo of IsOK establish , The page access is successful , Otherwise, the page access fails
    8. // Encounter failure - You can do it yourself , For example, re request or write LOG
    9. richTextBox1.Text = ho.TxtData;
    10. }
    11. else
    12. {
    13. richTextBox1.Text = " Page access error ";
    14. }
    15. // Finally, destroy the object
    16. hi.Dispose();

You can pretend to be whatever you need , For example, wechat , The unique information of Momo .
2:Cookie
SuperWebClient Default processing WEB From the server COOKIE, You don't have to think about it yourself , That's from the server COOKIE How to read information

HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
if (ho.IsOK)
{
richTextBox2.Text = ho.Cookies; // If HO The logo of IsOK establish , The page access is successful , Otherwise, the page access fails
// Encounter failure - You can do it yourself , For example, re request or write LOG
richTextBox1.Text = ho.TxtData;
}

In the output object ho.Cookies In the attribute .

For example, visit C# Ant nest  http://www.csharpworker.com/

To get the cookies yes
hTQ9_2132_pc_size_c=0;hTQ9_2132_onlineusernum=2;hTQ9_2132_lastact=1488261049%09forum.php%09;hTQ9_2132_sid=mRojS6;hTQ9_2132_lastvisit=1488257449;hTQ9_2132_saltkey=p7EgLlZg;
WEB The server will send different cookies, SuperWebClient Output object of httpoutput It's just sent by the server at the time of the request COOKIES Information , It doesn't accumulate , although SuperWebClient Internally, all the messages sent from the server will be accumulated COOKIE And send it back when it is submitted to the server , But it's not available to read all the cumulative COOKIE Methods , You can collect feedback on each request by yourself httpoutput Medium COOKIES Information , generally speaking , I don't pay much attention to this thing , Unless you want to save COOKIES Do it another way .

3:POST Sign in
         In general , If no accident ,WEB Many operation requests of services are based on POST Agreed ,AJAX It's also , Very little is based on GET Agreed , Because the request string is length limited
A lot of parameters need to be passed through POST To send the , Next , Let's see. SuperWebClient How to use POST How to log on to a website , Before the show , We need to analyze the website login
Request agreement for , And then use SuperWebClient To simulate .
Let's explain it with the process of logging into the blog park , First of all, we use Fiddler Capture the login protocol of blog park , Be careful , If you can't get the login protocol of the blog Park , See below 2 individual FIDDLER Two of the most common problems in , Because blog Park login is adopted HTTPS, So it needs to be turned on FIDDLER Corresponding HTTPS Decoding options .
Click on FIDDLER Menu TOOL->TERERIK  FIDDLER OPTIONS

If you come across something you can grab , But it's hard to decode , That's your certificate problem , Certificate problem - You can refer to http://blog.csdn.net/chjqxxxx/article/details/53666175  here - Rebuild the certificate - The problem will be solved

Here is the login protocol of the blog park

POST https://passport.cnblogs.com/user/signin HTTP/1.1
x-requested-with: XMLHttpRequest
Accept-Language: zh-cn
Referer: https://passport.cnblogs.com/user/signin?ReturnUrl=http%3A%2F%2Fwww.cnblogs.com%2F
Accept: application/json, text/javascript, */*; q=0.01
verificationtoken: _Q9B84azUZnUTIffmygxm8pONeAnXanp7gw7sqep3b3Lqk-opvj5wPgeAN6SM6lN1qFi6y8-36GAyjMDYYBuf_cy6iE1:fmSuVFj_iIGBnMV7N3TydHUuc8ORlIkNw7eSCsGqwV2BX7lM64bNoMV0X3S-t2-rlDcBpfuYnK3_In_fMNTFaPhE3Cw1
Content-Type: application/json; charset=utf-8
Accept-Encoding: gzip, deflate
User-Agent: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; .NET4.0C; .NET4.0E)
Host: passport.cnblogs.com
Content-Length: 386
Connection: Keep-Alive
Cache-Control: no-cache
Cookie: Hm_lvt_cc17b07fc9529e3d80b4482c9ce09086=1482645471; _ga=GA1.2.806287383.1482645352; __gads=ID=cde93307386572b7:T=1488262791:S=ALNI_MbB1oiDaPhuYt-REwWv3uYFQG8-AA; AspxAutoDetectCookieSupport=1; SERVERID=227b087667da6f8e99a1165002db93f6|1488285061|1488285061 {"input1":"a9Q+6M4EwC0HlHafyL8GYvd5zcFP+tUMV+F2P+aF0FdzrtY9nqPdkjCDctPQlY37EI/VzpLc9z+SAKXaWpWa0yDUM3USN+b7W+m6t7p8qRS+nuHu1G6wJ34IyuAruQ5gE3+r30PKy65q9dD0F5ruOSZizIYgCvm2/WrQfQsdYiQ=","input2":"fGOIEUEG5HP6tt3c/q9z0DuLyc9qeq0GiJQyTGNhTL/sCriRomR2G8gtn+AurDFzH/dDzMOYrBfEJ97n454gg8Ppd5cRe3qaNBTPebCJrIK3chv/Mupe+liFvEu6MwOgPqFBRm6GQCSZBHaVfRpHBKYrQkEaWRZ6ItSjKCBKpQc=","remember":false}

The account number and password are encrypted , As for the encryption process , You need to carefully analyze the blog park's JS Code , Here we are just to illustrate how to use it SuperWebClient Conduct
POST Form login , So for the account number , The cryptanalysis of the password is omitted , If there's a lot of free time in the back , I will add an article to analyze the account number of the blog park , Password encryption , Mainly related to
JS Debugging and analysis of code .

So use SuperWebClient How to log in to the blog Park . It's simple .

HttpInput hi = new HttpInput();
HttpManager.Instance.InitWebClient(hi, true, 60, 60);
// Set up blog Park login here URL
hi.Url = "https://passport.cnblogs.com/user/signin";
hi.Refer = "https://passport.cnblogs.com/user/signin?ReturnUrl=http%3A%2F%2Fwww.cnblogs.com%2F";
// We forge down the road , Prove where we're from , According to the agreement
// POST data hi.PostData = "input1=a9Q+6M4EwC0HlHafyL8GYvd5zcFP+tUMV+F2P+aF0FdzrtY9nqPdkjCDctPQlY37EI/Vzp"
+"Lc9z+SAKXaWpWa0yDUM3USN+b7W+m6t7p8qRS+nuHu1G6wJ34IyuAruQ5gE3+r30PKy65q9dD0F5ruOSZizIYgCvm2/WrQfQsdYiQ="
+"&input2=fGOIEUEG5HP6tt3c/q9z0DuLyc9qeq0GiJQyTGNhTL/sCriRomR2G8gtn+AurDFzH/dDzMOYrBfEJ97n454gg8Ppd5cRe3qa"
+"NBTPebCJrIK3chv/Mupe+liFvEu6MwOgPqFBRm6GQCSZBHaVfRpHBKYrQkEaWRZ6ItSjKCBKpQc="
+"&remember=false"; HttpOutput ho = HttpManager.Instance.ProcessRequest(hi);
if (ho.IsOK)
{ richTextBox1.Text = ho.TxtData;
}
else
{
richTextBox1.Text = " Page access error ";
} // Finally, destroy the object
hi.Dispose();

hi.PostData=" Set up POST The form data "       The format is :   key1=value1&key2=value2..... In turn

Today's topic is over , With some of today's stuff , You've been able to handle most of the mock requests ,SuperWebClient Most of the features of are also around the present GET,POST service .

SuperWebClient - One is based on CURL Of .NET HTTP/HTTPS Analog God component (2) More articles about

  1. SuperWebClient - One is based on CURL Of .NET HTTP/HTTPS Analog God component (1)

    We all know , Whether you're a reptile or not , It's a collection tool , Their HTTP/HTTPS Mock access is always a fundamental issue , I guess there are a lot of people like me , be that as it may , That kind of built-in or third-party class library uses a lot of , But there are always some unsatisfactory problems , also ...

  2. One is based on nodejs, Support http/https The broker in (MITM) agent , Easy for penetration testing and development debugging .

    Source code address :https://github.com/wuchangming/node-mitmproxy node-mitmproxy node-mitmproxy It's based on nodejs, Support http/h ...

  3. Hand in hand write one based on Spring Boot Parameter verification component under the framework (JSR-303)

    Preface In the process of the research and development of the new open platform , Because different interfaces need to verify different input parameters , This involves the verification and encapsulation of general parameters , If it's not packaged , Then the verification code will be written in different styles . The verification tool class is inconsistent . High maintenance risk and other factors ...

  4. Yoshino: One is based on React Can be customized PC Component library

    Github: https://github.com/Yoshino-UI... Docs: https://yoshino-ui.github.io/#/ Cli-Tool: https://git ...

  5. 【 original 】 It's based on .NET Of Redis Cluster Cluster access components

    Hello Hello everyone , I am a TANZAME, We meet again . Today we're going to talk about how to make one  Redis Cluster Cluster clients , There are dry goods by hand , Your fine products . As the business grows , Online environment QPS bulge , Naturally, the current single ...

  6. Simulated Login artifact PHP be based on cURL Implementation of automatic simulation Login Class

    One . conceived from Firefox Browser copy cURL command ( Initial page . Submit . After submission ) Automatic analysis curl Form a simulated login code Default parameters :ssl/302/gzip Two . Realization Interface ( One ) according to curl Information execution and parsing results p ...

  7. ordinary php be based on curl The reverse agent for

    cause : The manager : You need to implement a reverse proxy ? I :   Simple ,nginx Set it up every minute . The manager : Um. ? No, nginx? I : nodejs That's fine , There are examples on the Internet that can be done in minutes . The manager : Um. ? Only virtual hosts , Can only upload php Program ? ...

  8. SDKMAN A command line interface based SDK User environment manager

    1. background Have used Python Friends of development , It should be understood that Python2 and Python3 Grammatical differences , Sometimes you download code from the Internet based on different interpreters , To switch versions back and forth , It's not very convenient to use , Sometimes it's even troublesome . So someone invented ...

  9. psutil One is based on python Cross platform system information tracking module

    Thanks to this module , Here I recommend a hand . https://pythonhosted.org/psutil/#processes psutil It's based on python Cross platform system information monitoring module . stay pytho ...

Random recommendation

  1. ik Extended support Solr To configure

    Expand ik Native code :public class IKAnalyzerTokenizerFactory extends TokenizerFactory{ private boolean useSmart; ...

  2. About PHP Of curl Open the question

    I'm doing a Sina login function today , Sina offers PHP SDK You need to use curl_init() function , It can't be found during debugging curl_init() Methods .  We need to open it after searching curl Of php Expand , that curl And what is it ...

  3. Javascript The operation table is interlaced

    <!DOCTYPE html> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <m ...

  4. python+robot framework Interface automation testing

    python+requests The request to implement the interface has been described in the previous article , You can also visit if you don't understand or have questions python+request Interface automation framework At present, we need to consider how to achieve keyword driven interface automation output , By keyword ...

  5. thorough Web Request process

    B/S Network architecture Benefits : 1. The client uses the same browser . -- The interactive nature of the browser makes it very easy to use 2. The server is based on a unified http.  -- simplify . Standardize the development mode , Greatly reduce development costs . Such as tomcat ngi ...

  6. npm Release vue Component flow

    Initialize project vue init webpack-simple XXX Define the components Release configuration 1.package.json 2.webpack.config.js( The comment section is the original configuration ) Release 1. Sign in 2. Release n ...

  7. https Self signed SSL certificate

    Introduce TLS Or transport layer security , And its predecessor SSL( Represents the secure socket layer ) Is used to wrap normal traffic in a protected encryption wrapper Web agreement . Using this technology , The server can safely send traffic between the server and the client , Without being intercepted by outside parties ...

  8. C# Use CefSharp Open source library development Chrome browser

    One . Introduce          I haven't touched this before , But it's useful in corporate projects , So I'll look at it in passing . I just did it today WinForm Test of , Try it when you have time WPF How is it realized . It was a mess at first , It's a bit numb ...

  9. About CoreData Usage of

    Some colleagues feel that CoreData It's a person who doesn't understand , Mysterious things that can't be understood , Actually ios Local data storage is a sqlite database , A simple database , And this CoreData Does it support all stored data , Obviously not , Standing in my corner ...

  10. upsource On

    stay JetBrains On our website , notice codereview Tools for upsource ,https://www.jetbrains.com/upsource/   Official English documents Let's take a look at the bloggers in the blog park ...