BDO Bot Scrapping BDOCODEX

Here you can ask your questions and post suggestions about site's development.
Post Reply
ayeven
Posts: 3
Joined:Sat Dec 26, 2020 7:18 pm
Discord:Ayeven#9725

BDO Bot Scrapping BDOCODEX

Post by ayeven » Sat Dec 26, 2020 7:41 pm

Hello to admin of BDOCODEX,

As the tile say, I have made bdo bot that used your website data. So the questions arise after I made it, yeah sound silly I know. Questions here;
  • Is it legal or allowed?
  • Does bdocodex have rate limit. I have limit my scrapping bot so it wont flood the traffic though.
  • Here is the bdo bot that uses the script on scrapping Github Repo
You can take a look at the code if it do any harm to bdocodex. Again thank you

User avatar
Kiriak
Site Admin
Posts: 495
Joined:Wed Sep 11, 2013 10:30 am
Discord:Kiriak#5590

Re: BDO Bot Scrapping BDOCODEX

Post by Kiriak » Sun Dec 27, 2020 8:09 am

Sure, you can do it. Just make sure that your bot doesn't hammers the site too much or it'll be blocked by the ddos protection. And use the data caching as much as you can.

ayeven
Posts: 3
Joined:Sat Dec 26, 2020 7:18 pm
Discord:Ayeven#9725

Re: BDO Bot Scrapping BDOCODEX

Post by ayeven » Wed Dec 30, 2020 4:39 pm

Changed the code already from using pupeteer to axios and using fetch request. So it should not hammering website

User avatar
Kiriak
Site Admin
Posts: 495
Joined:Wed Sep 11, 2013 10:30 am
Discord:Kiriak#5590

Re: BDO Bot Scrapping BDOCODEX

Post by Kiriak » Wed Dec 30, 2020 8:14 pm

Nice.

Experience
Posts: 1
Joined:Tue Sep 05, 2023 12:57 am

Re: BDO Bot Scrapping BDOCODEX

Post by Experience » Tue Sep 05, 2023 12:58 am

Hi Kiriak! I'm doing a similar thing and was wondering if you're still alright with other folks using your website data. Let me know!

User avatar
Kiriak
Site Admin
Posts: 495
Joined:Wed Sep 11, 2013 10:30 am
Discord:Kiriak#5590

Re: BDO Bot Scrapping BDOCODEX

Post by Kiriak » Tue Sep 05, 2023 8:02 pm

Well, in general I don't like it. But it's not like I can do anything about it right? People are still trying to scrape the site several times every week...

Sebastianucas
Posts: 1
Joined:Wed Mar 20, 2024 3:49 pm
Discord:toplad

Re: BDO Bot Scrapping BDOCODEX

Post by Sebastianucas » Thu Mar 21, 2024 2:46 pm

I was hoping you still don't mind people accessing the data from your website because I'm doing something similar.
space bar clicker

Post Reply