Skip to content

Getting Started

After you installed this package, the next step is to import the package into your code and start using the functions.

from botcity.plugins.crawler import BotCrawlerPlugin

As a demonstration of the library, let's build a simple example together that will make a request to BotCity's Youtube channel and fetch the number of subscribers.

Making the request

To make the request you must use the request method which takes as an argument a URL.

# Instantiate the plugin and enable JavaScript
crawler = BotCrawlerPlugin(javascript_enabled=True)

url = "https://www.youtube.com/c/BotCityComputerVisionAutomationRPA"

# Make the request
html = crawler.request(url)

Locating the element

Looking into the page source we can notice that the element holding the subscribers information has the attribute id as subscriber-count.

Based on that we can use the get_element_by_id from the HTML object and finally read its value.

Here is how we do it:

# This sets the current element on the HTML object to the one found
html.get_element_by_id("subscriber-count")

# Read the value into the subscribers variable
subscribers = html.value()

Complete code

Let's take a look into the complete code:

# Import the plugin
from botcity.plugins.crawler import BotCrawlerPlugin

# Instantiate the plugin and enable JavaScript
crawler = BotCrawlerPlugin(javascript_enabled=True)

url = "https://www.youtube.com/c/BotCityComputerVisionAutomationRPA"

# Make the request
html = crawler.request(url)

# This sets the current element on the HTML object to the one found
html.get_element_by_id("subscriber-count")

# Read the value into the subscribers variable
subscribers = html.value()

Pro Tip

This plugin allow you to use method chaining so the code above could be written as:

from botcity.plugins.crawler import BotCrawlerPlugin
url = "https://www.youtube.com/c/BotCityComputerVisionAutomationRPA"
id = "subscriber-count"
subscribers = BotCrawlerPlugin(True).request(url).get_element_by_id(id).value()

Next Steps

Check our examples and experiment with the API. Let us know where it can be improved.

Have fun automating!

Back to top