Scrapping + crawling is not synchronus #98

New issue

Closed

opened 2025-10-14 18:09:53 -06:00 by navan · 0 comments

navan commented

2025-10-14 18:09:53 -06:00

Owner

Originally created by @AutomataVM on 3/19/2018

Subject of the issue

When I scrape a collection of results, I want it to start crawling to the next page after it finishes scraping all the current data. However, I find that when it executes, crawling happens before showing scrapped data

Your environment

version of node: v6.11.0
version of npm: v3.10.10

Steps to reproduce

function scrapeAndCrawl(searchUrl) {

if (!searchUrl){
	formData.splice(0,1);
	startSearch();

*Originally created by @AutomataVM on 3/19/2018* ### Subject of the issue When I scrape a collection of results, I want it to start crawling to the next page after it finishes scraping all the current data. However, I find that when it executes, crawling happens before showing scrapped data ### Your environment version of node: v6.11.0 version of npm: v3.10.10 ### Steps to reproduce function scrapeAndCrawl(searchUrl) { if (!searchUrl){ formData.splice(0,1); startSearch();

navan

2025-10-14 18:09:53 -06:00

closed this issue
added the
tech support

tech support

tech support

tech support

tech support

tech support
labels

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: github/x-ray#98

No description provided.

Rows
Columns