Script To Scrape Sales Nav Search Results

Andrew Pierno

Jan 3, 2023 • 4 min read

Will keep this super crisp, to the point.

Problem statement: Sales nav is great but just does not allow you to export the results as a CSV.

How to scrape search results?

Step 1

Go to sales navigator. Input the search query. The page url should look like https://www.linkedin.com/sales/search/people?recentSearchId=xxx&sessionId=xxx

Step 2

Open the browser developer console (famously known as inspect element window). Use the keyboard shortcut Ctrl + Shift + J (or Command + Option + J on macOS). This will open the developer console in a separate window.

Step 3

Paste the code below to the console. Press Enter.

Note: Change the variable NUM_OF_PAGES to the count of pages you'd like to scrape, starting from the current page. Assign NUM_OF_PAGES to 5 if you'd like to scrape 5 pages.

We highly recommend to not scrape a lot of pages in one go. Linkedin is very strict with banning accounts owing to increased usage of automation tools these days.

const NUM_OF_PAGES = 3;
const WAIT_BEFORE_EACH_PAGE = 5000;

const goToNextPage = () => {
  const nextButton = document.querySelector("[aria-label=Next]");
  if (nextButton) {
    nextButton.click();
  }
};

const scrollToBottom = () => {
  document.querySelector("#search-results-container").scrollTo({
    top: document.querySelector("#search-results-container").scrollHeight,
    behavior: "smooth",
  });
};

const jsonToCsv = (jsonData) => {
  const keys = Object.keys(jsonData[0]);
  const csvRows = [];

  csvRows.push(keys.join(","));

  for (const row of jsonData) {
    const values = keys.map((k) => row[k]);
    csvRows.push(values.join(","));
  }

  return csvRows.join("\n");
};

const downloadCsv = (csv) => {
  const blob = new Blob([csv], { type: "text/csv" });
  const url = window.URL.createObjectURL(blob);
  const a = document.createElement("a");
  a.style.display = "none";
  a.href = url;
  a.download = "data.csv";
  document.body.appendChild(a);
  a.click();
  window.URL.revokeObjectURL(url);
};

const delay = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

const cleanedString = (str) => {
  return JSON.stringify(str?.replace(/(\r\n|\n|\r)/gm, "")?.trim());
};

const runScraper = async () => {
  let data = [];

  for (let page of Array(NUM_OF_PAGES).keys()) {
    console.log("Scraping page: ", page + 1);

    await delay(WAIT_BEFORE_EACH_PAGE);
    scrollToBottom();
    const nodes = document.querySelectorAll(".artdeco-list__item");
    await delay(2000);

    nodes.forEach((node) => {
      data.push({
        name: cleanedString(
          node.querySelector("[data-anonymize=person-name]")?.innerText
        ),
        title: cleanedString(
          node.querySelector("[data-anonymize=title]")?.innerText
        ),
        location: cleanedString(
          node.querySelector("[data-anonymize=location]")?.innerText
        ),
        companyName: cleanedString(
          node.querySelector("[data-anonymize=company-name]")?.innerText
        ),
        linkedinUrl: JSON.stringify(
          node.querySelector(
            "[data-control-name=view_lead_panel_via_search_lead_name]"
          )?.href || ""
        ),
        imgUrl: JSON.stringify(node.querySelector("img")?.src || ""),
        page: JSON.stringify(page + 1),
      });
    });
    await delay(1000);
    if (page < NUM_OF_PAGES - 1) {
      goToNextPage();
    }
    await delay(2000);
  }
  console.log(
    `Scraping Completed! ${data.length} results found. Downloading CSV...`
  );
  const csv = jsonToCsv(data);
  downloadCsv(csv);
};

runScraper();

Step 4

Wait for it to scrape the results and the csv to start downloading.

Step 5

Import the CSV into Google Sheets.

Some tricks to refine your search results (apart from the filters).

Exact phrases
Simply enclose the phrase with double quotes ("") to make sure to include only return profiles with that exact phrase in the headline, summary, or job title in the search results.
Union phrases
Use the OR operator to broaden your search: For example, searching for "software engineer" OR "developer" will return profiles that have either term in the headline, summary, or job title.
Intersection phrases
Use the AND operator to narrow your search: For example, searching for "software engineer" AND "San Francisco" will only return profiles that have both terms in the headline, summary, or job title.
Unwanted phrases
Use the NOT operator to exclude certain terms from your search: For example, searching for "software engineer" NOT "San Francisco" will return profiles that have "software engineer" in the headline, summary, or job title but not "San Francisco."
Priority
Use parentheses to group search terms: For example, searching for (software engineer OR developer) AND (San Francisco OR Los Angeles) will return profiles that have either "software engineer" or "developer" in the headline, summary, or job title and either "San Francisco" or "Los Angeles" in the location field.

How to automate Linkedin outreach?

Super Send helps you automate Linkedin messages, connection requests, and profile views.

Create a campaign.

Connect your Linkedin Account

Download our chrome extension:

https://chrome.google.com/webstore/detail/super-send/gdebmlmbenonapffoingafcjanfhjgbf?hl=en-US

Once you log in to linkedin, it will grab the credentials of whomever you are currently logged in as.

If you need to attach more than one profile, try chrome profiles to add multiple users.

Simply upload the leads with the sales nav or the Linkedin profile url.

Start building your sequence by adding events like "Linkedin Profile Visit", "Linkedin Connection with Message", "Linkedin Message". You can even add follow up emails or a twitter dm based on different if else statements. Multi-channel for a reason.