Need to design a web crawler using C language, the specifications for this assignment is in that project 1 web crawler pdf. There’s two additional documents for starting tips.
My tutor said the sort of http libraries that you shouldn’t use are libcurl. (as an exception, you can use the HTML parser htmltidy.c from libcurl.)
Also they said client.c could be a good starting point, most of them actually use that file to modify to make it work as a crawler.
All applicants go through a series of tests that check their level of English and knowledge of formatting styles. The applicant is also required to present a sample of writing to the Evaluation Department. If you wish to find out more about the procedure, check out the whole process.