ae700330b9
When user enters something invalid (like number 123), it wuold still hit FB webpage and return HTML. I do simple detection by looking at title when it attempts to parse the DOM directly. When it contains "Content Not Found", it is skipped. Non-existent web pages cannot be parsed by using LDJSON parser and looking at null data. |
||
---|---|---|
.. | ||
crawler.spec.js | ||
dom-parser.spec.js | ||
ics-retriever.spec.js | ||
ldjson-parser.spec.js |