Finding a house to rent with R

I find myself in a real world situation where web scraping will be useful. I’ve never tried it before, this will be my first go at getting something happening.

My objective is to collect information from realestate.com.au to help me understand the market before moving to Adelaide.

I’ll be using the ‘rvest’ package and hacking my way through while reffining to a couple of tutorials here and here.

library(tidyverse) # For everything
library(rvest) # For some prettier themes

Scraping a single page

I start with a simple rental search for properties between 300 and 450 per week with minumum 3 bedrooms.

url <- "https://www.realestate.com.au/rent/property-unitblock-villa-townhouse-unit+apartment-house-with-3-bedrooms-between-300-450-in-adelaide,+sa+5000/list-1"
webpage <- read_html(url)

I will start with weekly Price which is defined as .priceText

price_data_html <- html_nodes(webpage,'.priceText')

price_data <- html_text(price_data_html)

head(price_data)
## [1] "$340 per week"    "$400 per week"    "$750 per week"   
## [4] "$450 per week"    "$420 - $440"      "$385 - $400 / Wk"

Then the number of bedrooms, bathrooms and carspaces which is simply dd

beds_data_html <- html_nodes(webpage,'dd')

beds_data <- html_text(beds_data_html)

head(beds_data)
## [1] "3" "1" "1" "3" "1" "1"

Refining the elements

Running the scraping over all items