Whenever you call the response function of the skrape{it} DSL it will take the given request config and performs a HTTP request by the use of the given fetcher.
Example
1
skrape(AsyncFetcher){// <-- could use any valid fetcher depending on your use-case
2
request {
3
// some request config
4
url ="https://some.url"
5
userAgent ="my fancy UA"
6
}
7
8
response {// 🍉 <-- response will make actual HTTP request and convert its response to a Result object
9
// the scope of 'this' inside the response lambda is Result!
10
// that means you can directly call any field or function of Result here.
11
12
// e.g. print the response body
13
println(responseBody)
14
15
// the last thing called inside the response lambda will be returned
16
// e.g. the status code
17
status { code }
18
}
19
}
Copied!
The response of a call done by a fetcher will be represented in a Result object.
A result comes with a bunch of basic fields as well as some handy DSL functions
Fields
Name
Description
Type
responseBody
The response body as raw String representation. At this point it can be everything from HTML to XML, JSON or even plain text.
The baseUri argument is used to resolve relative URLs into absolute URLs, and should be set to the URL where the document was fetched from and will be set based on the given request URL.
String
The html-parser artifact provides extension functions to directly parse the responseBody String to Html document (Doc object) that represents the html tree based on DocElements .