In this guide, we're going to create screenshots for a list of LinkedIn URLs in Airtable. We'll use the Data Fetcher Airtable extension and the Urlbox screenshot service. By default, trying to screenshot a LinkedIn page/ profile URL will just create a screenshot of the LinkedIn login screen, so we'll need to use a cookie to make LinkedIn think we're logged in. The screenshots we create from the LinkedIn URLs will be stored in an Airtable attachment field.
Create a table in your Airtable base called 'Pages'.
Then create a field in this table called 'URL' with the type 'URL'. Add all the LinkedIn URLs you want to screenshot, one in each record. If you've already got a list of URLs, you can ignore this step.
Then create a field called 'Screenshot' with the type 'Attachment'.
In the same table, create a new Grid view called 'Needs screenshot' with the following filters:
The records in the 'Needs screenshot' view are what we'll run Data Fetcher on. As the screenshots are created, they will move out of this view.
Data Fetcher is a free extension that lets you connect to other applications like Urlbox, Google Maps and lots more right inside Airable. Add Data Fetcher to your base from the Airtable marketplace.
After you have added Data Fetcher to your base, you will need to create a free account. Signing up for a Data Fetcher account keeps your details secure. It's separate from your Airtable account.
Enter a password and click 'Sign up for free' or click 'Continue with Google'.
Requests in Data Fetcher are how you send data to/from your Airtable base. We're going to create a request to connect to the Urlbox service. On the Data Fetcher home screen, click 'Create your first request'.
On the next screen, for Application, select 'Urlbox'.
Paste this API key into the API Key input in Data Fetcher (under Authorization).
For Endpoint, select 'Create screenshots from URLs'.
Enter 'Import LinkedIn Screenshots' for the request Name. Then click 'Save & Continue'.
For URL, we want to insert a reference to our 'URL' field. This means Data Fetcher will use the values from our 'URL' field in Airtable when we run the request. First, click the + button on the right-hand side of the URL input in Data Fetcher.
A modal will open. Select 'URL' for Field and 'Needs screenshot' for Run for every record in view. Then click 'Confirm'.
The URL field will now contain a reference to the table.
Select the Format you want to use for the screenshots, e.g. 'PNG'.
We need to make LinkedIn think we are logged in when we capture the screenshot. Otherwise, we will just get a screenshot of the login screen!
To do this, sign in to LinkedIn, open your browser's network tab and navigate somewhere in LinkedIn, e.g. someone's profile. Then go to the network tab, click on a request, then 'Cookies' and copy the value for the li_at cookie.
If you don't follow this step, you can watch the YouTube video that goes along with this article.
Back in Data Fetcher, click 'More options for this Urlbox endpoint'.
Scroll to Cookie. Enter the following, replacing
YOUR_LINKEDIN_LI_AT_COOKIE with the LinkedIn cookie you just got from your browser:
Make sure the Output Table & View is set to 'Pages' and 'Needs screenshot'.
Click 'Save & Run'.
The request will run and the Response field mapping modal will open. Make sure the 'Screenshot' field from Urlbox is mapped to your existing 'Screenshot' field in Airtable. Then click 'Save & Run'.
Data Fetcher will then run the request for every record in 'Needs screenshot' and convert the LinkedIn page URL to screenshots in Airtable as if you are logged into LinkedIn!
At the moment, we have to manually run the request to create the screenshots. We can use Data Fetcher's scheduled requests feature to automatically create them every 15 minutes/ hour/ day etc.
In Data Fetcher, scroll to Schedule and click 'Upgrade'.
A new tab will open where you can select a plan and enter your payment details to upgrade.
Return to the Data Fetcher extension and click 'I've done this'.
Under Schedule click '+ Authorize'.
A window will open where you'll be prompted to authorize which Airtable bases you want Data Fetcher to have access to.
We recommended selecting 'All current and future bases in all current and future workspaces' to avoid any issues with unauthorized bases in the future.
Click 'Grant access'.
Back in Data Fetcher, you'll see Schedule this request is now toggled on.
Select how often you want the request to run, e.g. 'Every hour'. Click 'Save' The request will now convert the LinkedIn URLs to screenshot files on this schedule.
Oct 15, 2023•
Rosie Threlfall•LinkedInWeb Scraping