Can I start adding these shiny new entries to my collections? or should i wait until everything is properly settled? I don’t want to transfer my completion dates if it’s all going to be deleted afterwards.
There are now separated entries for each of the crash bandicoot remakes, which I prefer rather than the whole collection as one thing.
You can go ahead and use them. If you want to wait, your completion dates will automatically get transferred over to the IGDB entry once they get merged. The only way to do the dates right now is on your shelf page, so it’s a little bit cumbersome to do it. I’ll get all the functionality added back to the new pages soon. I have to get my basement emptied out tomorrow, but I’m going to try and steal a couple hours to work on this.
Sorry, I’ve been slow to respond to everyone in this thread! I am reading everything.
Towards the end, I would maybe get 10 - 15 games changed per 8 hour pull.
I can’t find anything either! Luckily their Discord is good, and they’re pretty good about responding to emails. I’ll just probably start bugging them about it soon.
This is probably the right answer. Their API only allows you to get 500 games in 1 call. Even I dumped them all in a queue, I could get to them at a little slower pace. They do have data dumps that you can grab a CSV file off their server too. I need to go back and re do my partner agreement with them before I can do that though I believe.
I’ll send you a message here this weekend. You should be able to see it all on the admin site with the super powers you have. If you also search for a game on the site with the exact name, the IGDB entry will show up in the list. So if you search for Super Smash Bros. for Wii U or Pokémon Red Version, you’ll see the IGDB entries show up.
Talking from a user point of view, I think you could time out updates based on release date. Like you said, if a rating of SMB for the NES counts as an update, could you not just tell it to only update games 30 years out once every two months or more?
Live updates would really only be useful for games on release.
Also definitely want my ‘Super Curator’ Badge.
Whoa, I discovered this had started to happen by accidentally searching the full title of The Thaumaturge: Deluxe Edition and landing on the new IGDB-style page! Nice!
Cool - I also mean, how does one enter a review, playtime, etc. for an IGDB entry?
So, we will see many many duplicated records for a while? Such as - Search results for "Life After Magic"
(also that one is weird because they have two different release dates).
Looks like Giantbomb has the original Itch release date (of the incomplete game), whereas IGDB uses the later Steam release date (of the complete game)…
I guess there will be a fair bit of work ahead to correct such discrepancies!
I just got access to the CSV file dumps from IGDB yesterday. I’m working on some code to process those instead of the way I’m updating currently. It should go a lot quicker and more efficiently to be able to process a big file like that instead of making a whole bunch of API calls that. Once I get that done, I’ll get to finishing up the IGDB pages, then I’ll get to the merging. Super easy 
I’m 80% done with my application, those last 20% are going to be a breeze.
Haha. That’s my whole life. The other night I needed to do a “quick” gutter job on my house. I was just going to cut the aluminum gutter off about 18" off the ground, add the corrugated pipe adapter, and then attach the corrugated pipe. Should be a 20 minute job that I’ve done several other times. The whole stupid gutter fell apart once the ground wasn’t holding it up. I had to screw it together at every joint, remount it to the garage, etc. Turned into at least an hour.
Hopefully all this computer stuff is a breeze compared to that
Thank you for keeping this place running and not abandoning it. I was telling my partner how sad it would be to watch this place get lost.
I don’t think that’s necessarily the case. The IGDB contribution guidelines say
A release date is the date that a game becomes playable for the very first time to the general public (according to platform and regions). We’ll always go with the earliest date first, (for example beta, early access or game preview), as long as they’re playable by everyone.
I had the same question about this a while ago for giantbomb and asked for clarification on their forums:
Early access date should always be the first release date on the main page, for example Hades has a release date of December 7, 2018 when it came out in Early Access but its finished release date was September 17, 2020.
So data quality and edge cases notwithstanding, the initial release on both sides should be the data where people generally have access to a game.
Ah, interesting to know! I was only describing an observation about that particular discrepancy @WerqKween noticed rather than guessing at their policies in general, which I wasn’t aware of. So I guess it was simply an error on the IGDB page in that case.
Yeah, my take is it should be the earliest playable release.
Even though those are the IGDB guidelines, I don’t think they are being followed currently. At least not in the games I’ve checked.
Wartales: Early access release in 2021, full game in 2023, and the release date that is front a center is the 2023
Same with Deep Rock Galactic, ea in 2018, full release in 2020, considered a 2020 game in the database
Slay the spire, ea in 2017, full release in 2019, considered a 2019 game
The fricking funny cube game, first public alpha in 2009, full release 2011, considered a 2011 game by IGDB.
I feel like we should use the full release date as THE release date for a game, just for convenience sake, it looks cleaner and more importantly changing them one by one would be a pain.
You’re absolutely right. Data can be inconsistent on IGDB, which is to be expected with a database driven by community submissions with a relatively minimal vetting process, especially at this scale. That’s what I was referring to when I mentioned data quality.
The guidelines on IGDB are fairly unambiguous, and the topmost priorities for any factual database should always be quality and consistency, not opinion, which is why I avoided stating how I believe things should be handled at IGDB.
If entries are not aligned with the guideline it means they are incorrect and should be updated. If the guidelines do not accurately reflect lived reality of how data is handled across IGDB as a whole, then they should be reviewed to decide if they need to be changed - but until that happens, entries inconsitent with the current guidelines should still be considered incorrect.
For the grouvee side of thing, other than user-specific data like ratings, completion times, etc. I’m not sure there is a strong need to migrate anything between game entries from different data sources. If existing entries from giantbomb conflict with newer ones from IGDB, the newer entries can simply replace the old ones.
I wish there was room for both the initial release and 1.0 release tbh.
I have not checked to see how accurate it is in IGDB, but their data structures are setup to handle that. I don’t have Grouvee setup to handle it correctly, but I could if I think their data is accurate enough for it. Basically they have a first release date field which theoretically should be the first playable date period, but then they have a whole set of release dates with categories you could store too. So theoretically an early access game, like Hades, should have the first release date (which is what I use here on Grouvee) set to whatever the first playable date is. Then it would have other release dates with statuses set to Full Release. In this example, their release date is set to the Full Release date, so I don’t know.
Not related to release dates, I’ve been working on adapting my update scripts to use their CSV files. It’s a little slower than I’d like, but I finally made a big breakthrough. Once I’ve got that working consistently, I’ll get back to work on finishing up the IGDB pages.
I had to upgrade the servers on Grouvee this weekend as well. I doubled the size as well as the cost. I could really use some more Grouvee Gold support. I’ll probably hammer this some more as I get more IGDB support added.
Thanks again for sticking with me!