skrape{it}
1.1.X
1.1.X
  • Introduction
  • Why it exists
  • overview
    • Setup
    • Who should be using it
  • Http Client
    • Overview
    • Fetchers
      • HttpFetcher
      • BrowserFetcher
      • AsyncFetcher
      • Implement your own
    • Request Options
    • Pre-configure client
    • Response
      • Status
      • Cookies
  • Html Parser
    • Parsing HTML
  • assertions
    • expect content
  • How to Use
    • Testing
    • Scraping
    • JS-rendered sites
  • Examples
    • Grab all links from a Website
    • Creating a RESTful API (Spring-Boot)
  • GitHub Repo
  • Extensions
    • MockMvc
      • Getting Started
      • GitHub Repo
    • Ktor
      • Getting Started
      • GitHub Repo
  • About skrape{it}
Powered by GitBook
On this page

Was this helpful?

  1. Examples

Creating a RESTful API (Spring-Boot)

This example assumes a working Spring-Boot setup.

This is a basic example on how you could build a REST API directly return data from a scraped website as JSON:

import it.skrape.extract
import it.skrape.selects.elements
import it.skrape.selects.element
import it.skrape.skrape
import org.springframework.web.bind.annotation.GetMapping
import org.springframework.web.bind.annotation.RestController

@RestController
class MyController {

    @GetMapping("/user-data")
    fun extractGithubUserData() =
        skrape {
            url = "https://github.com/skrapeit"
            extract {
                MyScrapedData(
                    userName = element(".h-card .p-nickname").text(),
                    repositoryNames = elements("span.repo").map { it.text() }
                )
            }
        }
}

data class MyScrapedData(val userName: String, val repositoryNames: List<String>)

This will return the following JSON as response body when /user-data is getting called:

{
  "userName": "skrapeit",
  "repositoryNames": [
    "skrape.it",
    "skrapeit-docs",
    "skrapeit-ktor-extension",
    "skrapeit-mockmvc-extension",
    "skrapeit-parent"
  ]
}
PreviousGrab all links from a WebsiteNextMockMvc

Last updated 6 years ago

Was this helpful?