skrape{it}
1.1.X
1.1.X
  • Introduction
  • Why it exists
  • overview
    • Setup
    • Who should be using it
  • Http Client
    • Overview
    • Fetchers
      • HttpFetcher
      • BrowserFetcher
      • AsyncFetcher
      • Implement your own
    • Request Options
    • Pre-configure client
    • Response
      • Status
      • Cookies
  • Html Parser
    • Parsing HTML
  • assertions
    • expect content
  • How to Use
    • Testing
    • Scraping
    • JS-rendered sites
  • Examples
    • Grab all links from a Website
    • Creating a RESTful API (Spring-Boot)
  • GitHub Repo
  • Extensions
    • MockMvc
      • Getting Started
      • GitHub Repo
    • Ktor
      • Getting Started
      • GitHub Repo
  • About skrape{it}
Powered by GitBook
On this page

Was this helpful?

  1. Http Client

Pre-configure client

PreviousRequest OptionsNextResponse

Last updated 3 years ago

Was this helpful?

Sometimes you may want to reuse your to avoid code duplication or to archive better maintainability of your code base. Skrape{it} supports shared client configurations by default -- you just need to configure the client properties you want to reuse using the skrape{it} DSL without calling the response function.

Calling the request function will create a Request object which can then be stored to a variable.

The DSL is design to behave like fluent API, which means it will always return the return value of the last thing that was called inside the skrape lambda function.

val myPreConfiguredClient = skrape(HttpFetcher) {
    request {
        timeout = 10_000
        headers = mapOf("some-custom-header" to "some-value")
        followRedirects = true
    }
}

@Test
fun `can use preconfigured client straight away`() {

    myPreConfiguredClient.response {
        status { code toBe 200 }
        headers.getValue("some-custom-header") toBe "some-value"
    }
}

@Test
fun `can use preconfigured client but slightly modify`() {

    myPreConfiguredClient.apply {
        request {
            followRedirects = false
        }
    }.response {
        status { code toBe 301 }
        headers.getValue("some-custom-header") toBe "some-value"
    }
}
HTTP Client configuration