skrape{it}
1.1.X
1.1.X
  • Introduction
  • Why it exists
  • overview
    • Setup
    • Who should be using it
  • Http Client
    • Overview
    • Fetchers
      • HttpFetcher
      • BrowserFetcher
      • AsyncFetcher
      • Implement your own
    • Request Options
    • Pre-configure client
    • Response
      • Status
      • Cookies
  • Html Parser
    • Parsing HTML
  • assertions
    • expect content
  • How to Use
    • Testing
    • Scraping
    • JS-rendered sites
  • Examples
    • Grab all links from a Website
    • Creating a RESTful API (Spring-Boot)
  • GitHub Repo
  • Extensions
    • MockMvc
      • Getting Started
      • GitHub Repo
    • Ktor
      • Getting Started
      • GitHub Repo
  • About skrape{it}
Powered by GitBook
On this page

Was this helpful?

  1. Http Client
  2. Fetchers

AsyncFetcher

A none-blocking Http-Client implementation

PreviousBrowserFetcherNextImplement your own

Last updated 3 years ago

Was this helpful?

Skrape{it} provides different fetcher implementations. All of the Fetchers behave different and you should pick them depending on your needs.

The AsyncFetcher is a none-blocking http client. It will send an HTTP-request (with , headers etc.) to a given url and returns a result that consists of the http response status as well as of the response headers and body in a none-blocking fashion by the use of coroutines. Thereby it can be convenient called from a suspend function.

It is fast, robust and result will look simular to what you would get when surfing a page in your browser with deactivated Javascript.

Thereby it is the best pick if you want to call websites in parallel or at least from a coroutine scope and if you just don't care about the excecution of a websites Javascript.

if you do not need skrape{it}'s full feature set you can use the AsynFetcher standalone by adding the following dependency to your project.

build.gradle.kts
dependencies {
    implementation("it.skrape:skrapeit-asyn-fetcher:1.1.5")
}
pom.xml
<dependency>
  <groupId>it.skrape</groupId>
  <artifactId>skrapeit-async-fetcher</artifactId>
  <version>1.1.5</version>
</dependency>
given request parameters