In this quick tutorial, we'll demonstrate how easy it is to use Google’s Indexing API in Airtable to help index your website. The Indexing API allows you to notify Google when pages are updated, added or removed and prompts Google to re-crawl specific pages on your webpage.
By using the Data Fetcher extension for Airtable, we can achieve this without any need for code.
You will need to have access to Google Search Console for the website you wish to index.
Please note: Google's current recommendation is to only use the Indexing API with JobPosting or BroadcastEvent embedded in a VideoObject. However, according to this post by RankMath, other types of pages will also work.
Create a URLs Table
First, set up a table called 'URLs' in Airtable. Create a field called 'URL' with the type 'URL'.
In the URL field, copy and paste the list of URLs you want to ask Google to index, either because they have new content or because they are entirely new pages.
Next, create a field called 'Indexed' with the type 'Checkbox'.
Create a new Airtable view called 'Needs indexing'. Add the following filters: 'URL' is not empty and 'Indexed' is a false/empty checkbox. This is the view that Data Fetcher will use to get the list of URLs to re-index.
Install Data Fetcher
Install Data Fetcher from the Airtable marketplace. After the extension launches, sign up for a free Data Fetcher account by entering a password and clicking 'Sign up for free'. If you already have an account, click 'Have an account?' at the bottom left of the screen. You can also sign up with your Google account.
Use Google Indexing API to Index Your Website
Click 'Create your first request' on the Data Fetcher home screen. Requests are how Data Fetcher interacts with external APIs and websites in order to import or send data.
On the create request screen in Data Fetcher, for Application, select 'Google Indexing' to use the Google Indexing API to Airtable integration.
Under Authorization, click on the + New Google Indexing connection button to connect your Google Account. This needs to be a Google account associated with Search Console for the URLs you want to index.
Next, authorize Data Fetcher access to your Google account.
Back in Data Fetcher, under Endpoint choose 'Notify Google that a page has been added, updated or removed'.
Give your request a name, such as 'Notify Google' and click 'Save & Continue'.
On the next screen, for URL click the + button to add a reference to the output table. For Field, select the 'URL' field in the table. Under Run for every record in view, select the 'Needs indexing' view. Then click 'Confirm'.
For Type select 'URL added or updated'.
Next, under Advanced Settings, scroll down to Add Fixed Value and set the 'Indexed' field to be a true/filled checkbox.
Click 'Save & Run'.
On the Response field mapping modal that opens, click 'Filter all' to filter out all the fields. Then, select the field called 'Latest update notify time' and map it to a new field. Then click 'Save & Run' in the bottom right. You will only need to set this step up once.
Data Fetcher will now run the request for every URL in the 'Needs indexing' view and notify Google that it needs to index these URLs. It will mark the 'Indexed' field as filled/true, which will filter the records out of that view. You can index a page again by unchecking the 'Indexed' checkbox and re-running the request.
If you look at the main view in your table, you will see the list of URLs, each of which has been sent to Google for indexing.
Automatically Index Web Pages in Airtable
At the moment, we have to manually run the request to connect to the Google Indexing API. However, by using Data Fetcher's scheduling feature, we can automatically check for content updates and re-submit our URLs to Google at regular intervals.
This is a paid feature, so you need to upgrade. In Data Fetcher, scroll to Schedule and click 'Upgrade'.
A new tab will open where you can select a plan and enter your payment details to upgrade.
Return to the Data Fetcher extension and click 'I've done this'.
Under Schedule click '+ Authorize'.
Next, a window will open where you'll be able to authorize the Airtable bases Data Fetcher can have access to.
By selecting 'All current and future bases in all current and future workspaces' you'll avoid having to authorize bases in the future.
Click 'Grant access'.
Back in Data Fetcher, you'll see Schedule this request is now toggled on.
Select how often you want the request to run, e.g. 'Every hour'. Click 'Save'. The Google Indexing Airtable integration will now check for new content and automatically request re-indexing on your chosen schedule.
