![]() log( "CHILD: url received from parent process", url) Ĭonst browser = await puppeteer. The code snippet below is a simple example of running parallel downloads with Puppeteer.Ĭonst downloadPath = path. □ If you are not familiar with how child process work in Node I highly encourage you to give this article a read. We can combine the child process module with our Puppeteer script and download files in parallel. Child process is how Node.js handles parallel programming. You can tap into this event and physically download the contents into a file. We can fork multiple child_proces in Node. This page downloads a csv by creating a comma delimited string and forcing the browser to download it by setting the data type like so let uri 'data:text/csv charsetutf-8,' + encodeURIComponent (content) window.open (uri, 'Some CSV') This on chrome opens a new tab. Our CPU cores can run multiple processes at the same time. □ Learn more about the single threaded architecture of node here Therefore if we have to download 10 files each 1 gigabyte in size and each requiring about 3 mins to download then with a single process we will have to wait for 10 x 3 = 30 minutes for the task to finish. This gave me a good excuse to try and automate the process as much as possible using puppeteer. ![]() It can only execute one process at a time. Puppeteer - login and video download dlw Puppeteer - login and video download Friday, 15 January 2021 6 min read TL DR I needed to download a video that was behind a login screen. You see Node.js in its core is a single-threaded system. However, if you have to download multiple large files things start to get complicated. ![]() In this next part, we will dive deep into some of the advanced concepts. ![]()
0 Comments
Leave a Reply. |