skrape{it}
1.1.X
1.1.X
  • Introduction
  • Why it exists
  • overview
    • Setup
    • Who should be using it
  • Http Client
    • Overview
    • Fetchers
      • HttpFetcher
      • BrowserFetcher
      • AsyncFetcher
      • Implement your own
    • Request Options
    • Pre-configure client
    • Response
      • Status
      • Cookies
  • Html Parser
    • Parsing HTML
  • assertions
    • expect content
  • How to Use
    • Testing
    • Scraping
    • JS-rendered sites
  • Examples
    • Grab all links from a Website
    • Creating a RESTful API (Spring-Boot)
  • GitHub Repo
  • Extensions
    • MockMvc
      • Getting Started
      • GitHub Repo
    • Ktor
      • Getting Started
      • GitHub Repo
  • About skrape{it}
Powered by GitBook
On this page

Was this helpful?

  1. Http Client
  2. Fetchers

HttpFetcher

A "classic" Http-Client implementation

PreviousFetchersNextBrowserFetcher

Last updated 3 years ago

Was this helpful?

Skrape{it} provides different fetcher implementations. All of the Fetchers behave different and you should pick them depending on your needs.

The HttpFetcher is a classic http client. It will send an HTTP-request (with , headers etc.) to a given url and returns a result that consists of the http response status as well as of the response headers and body. Nothing more, nothing less.

It is fast, robust and result will look simular to what you would get when surfing a page in your browser with deactivated Javascript.

Thereby it is the best pick if you want to call server-side rendered websites, a static html pages or if you just don't care about the excecution of a websites Javascript.

if you do not need skrape{it}'s full feature set you can use the HttpFetcher standalone by adding the following dependency to your project.

build.gradle.kts
dependencies {
    implementation("it.skrape:skrapeit-http-fetcher:1.1.5")
}
pom.xml
<dependency>
  <groupId>it.skrape</groupId>
  <artifactId>skrapeit-http-fetcher</artifactId>
  <version>1.1.5</version>
</dependency>
given request parameters