Page 1 of 1

BDO Bot Scrapping BDOCODEX

Posted: Sat Dec 26, 2020 7:41 pm
by ayeven
Hello to admin of BDOCODEX,

As the tile say, I have made bdo bot that used your website data. So the questions arise after I made it, yeah sound silly I know. Questions here;
  • Is it legal or allowed?
  • Does bdocodex have rate limit. I have limit my scrapping bot so it wont flood the traffic though.
  • Here is the bdo bot that uses the script on scrapping Github Repo
You can take a look at the code if it do any harm to bdocodex. Again thank you

Re: BDO Bot Scrapping BDOCODEX

Posted: Sun Dec 27, 2020 8:09 am
by Kiriak
Sure, you can do it. Just make sure that your bot doesn't hammers the site too much or it'll be blocked by the ddos protection. And use the data caching as much as you can.

Re: BDO Bot Scrapping BDOCODEX

Posted: Wed Dec 30, 2020 4:39 pm
by ayeven
Changed the code already from using pupeteer to axios and using fetch request. So it should not hammering website

Re: BDO Bot Scrapping BDOCODEX

Posted: Wed Dec 30, 2020 8:14 pm
by Kiriak
Nice.

Re: BDO Bot Scrapping BDOCODEX

Posted: Tue Sep 05, 2023 12:58 am
by Experience
Hi Kiriak! I'm doing a similar thing and was wondering if you're still alright with other folks using your website data. Let me know!

Re: BDO Bot Scrapping BDOCODEX

Posted: Tue Sep 05, 2023 8:02 pm
by Kiriak
Well, in general I don't like it. But it's not like I can do anything about it right? People are still trying to scrape the site several times every week...