Create a multithreaded parser tag titles
Employer
[no-member:pro]Vladimir[/no-member:pro]
Project parameters
Type of cooperationOne-time project
SectionWeb development
Prepaymentwithout prepayment
Payment methodsElectronic money
Acceptance of requestsfrom Jan 18, 2019 until Jan 28, 2019
Project description
I need a multithreaded parser tag title.
Multithreaded parser of content tags title list URL from txt.file.
Sites on the list may or may not exist.
Implementing is desirable in PHP or some server language to run with VDS.
If it works well under the wind, it does not matter what.
Settings:
- incoming data file
- outgoing data file
- 100-1000 streams
- 3-10 seconds timeout
-
- file with a step (if suddenly it stops, so as not to parse from the beginning, but from this step)
The parser should not download all 100% of the page code, but only from the characters specified in the settings.
For example, from the beginning of the file, we skip 183 characters and write only 34 characters into the variable, which are parsim.
Something like $info = file_get_contents ($url, NULL, NULL, 138, 34).
There could be 50 million in total in the incoming base. Links.
Multithreaded parser of content tags title list URL from txt.file.
Sites on the list may or may not exist.
Implementing is desirable in PHP or some server language to run with VDS.
If it works well under the wind, it does not matter what.
Settings:
- incoming data file
- outgoing data file
- 100-1000 streams
- 3-10 seconds timeout
-
- file with a step (if suddenly it stops, so as not to parse from the beginning, but from this step)
The parser should not download all 100% of the page code, but only from the characters specified in the settings.
For example, from the beginning of the file, we skip 183 characters and write only 34 characters into the variable, which are parsim.
Something like $info = file_get_contents ($url, NULL, NULL, 138, 34).
There could be 50 million in total in the incoming base. Links.
