This sounds amazing, I always preferred the IGDB way of sorting games, separating different versions, remakes and bundles. I might be a bit busy nowadays, but if there is anyway I can help you all, I’ll gladly lend a hand.
In my search for the “IMDB of games”, I found Backloggd which also uses IGDB, maybe you could get an idea of the final result from there? My only complaint is that there is too much stuff to browse through.
For example: Searching for Resident Evil 4 nets between 23 to 3260 results (IGDB search engine isn’t the best). You get separate entries for the gamecube version, the PS2 version, the Wii version, the HD port, the HD PC port, at least 4 collectors packages, the mobile version, and, of course, the VR version and remake. I believe some of those could be fused into a single entry and others deleted completely.
Anyway, I still like the idea, but if/when it happens, I’d like some forewarning, I wouldn’t mind an email even. That way, I’ll be able to keep track of any games that might get removed during the switch, so I can re-add them later.
I agree that this is going to be tricky to handle. IGDB is pretty good about tagging the category of a game, and it should give me the ability to mark very clearly that one version of Resident Evil 4 is the “main original entry” and that all of the others are some variation of remake/remaster/etc. I should be able sort by the category of game in search and hopefully not make it a mess. I think it’s necessary to break these games out like this though because people want to put the correct version of a game on their shelves I think.
There will be plenty of warning when the full switch is happening, although I don’t plan on removing any games. If a game is in our database, but not theirs, I’m not going to remove it. We might end up with a lot of duplicates that need to be merged, but I’d rather deal with that than removing data.
So IGDB has a big set of what they call “External Games” that basically maps an IGDB game to an external service. One of those external services is Giant Bomb, although I don’t think it’s advertised really. They have about 73,000 Giant Bomb to IGDB mappings. Behind the scenes I added an IGDB ID field to our database, and then started populating what I could using their mappings. There’s about 21,000 games that I don’t have a mapping for currently, and there were almost 1,000 duplicates where they had an IGDB game mapped to multiple GB games. I’m going to need some help with this when I can figure out a good way to surface this information
I’m pretty excited though. I like their API for the most part. I’m sure we’re going to have issues with obscure games, but hopefully we can give back to their dataset with all of our obscure game info once we get going.
This is going to take a while sorting all this data. I think I’m going to have to rethink how I’m doing this.
One idea I’m mulling around in my head is to just download every game from IGDB fresh and ignore Giant Bomb mappings to start off with. Then I’m thinking of building some type of interface where the community can help verify/suggest that an existing game in our database lines up with an IGDB entry. Once we verify that something lines up, we can move user shelf data over to the new game and get rid of the old one. The trick with this one is how would we handle search? At first, it would be a jumbled mess of a lot of games that are the same.
I’ve talked with a few of you via email, and have gotten some good suggestions. I’m still just feeling out the best way to handle this transition, so if anyone else has ideas, feel free to leave them here or send me a message.
You said you were tagging igdb games when working with the gb mappings. Can you tag the igdb entries before import so that searches exclude old entries and users can’t add old ones any longer for shelves? And then is there a way to automate the merge/replacing with igdb records by verifying the titles exact match and some other piece(s) of metadata match that confirms different entries with a same title? Then you’re left with a list of discrepancies and that’s all that has to be verified by hand or other method, instead of everything.
I’m not sure if I’m understanding right but I think a solution like the one devised by @Ceilort for his app (described in this forum post) would be a nice approach. Basically, mapping Grouvee entries to their respective IGDB entries by name matching, but in this case, additionally restricting search results to games in the “main game” category to avoid different versions of the same game popping up.
What I’ve done to start with is I’ve grabbed all of the IGDB to GB mappings that IGDB offers, and I’ve placed the IGDB ID in our database where I can. I’m confident in about 80-90% of them being accurate, which I don’t think is good enough. I do not want to just put my head in the sand and say, these are all good, let’s just grab the data from IGDB and overwrite what is there. I would hate for there to be a wrong mapping, particularly on a game with a lot of shelf entries, and all of a sudden everyone has a game on their shelves that isn’t what they thought it was. What I’m thinking of doing is wiping out all of those IGDB IDs, and just downloading all of the IGDB games as their own entries.
Following on from the last paragraph, I would tag all of these new IGDB entries as such, and I was thinking of doing exactly what you’re saying. Exclude old entries from search, and only let new entries be added to shelves. It would be a little weird because people would search for Breath of the Wild, go to the “new” page and be surprised that only 10 people have it on their shelf. It’s probably still worth it. The big games would switch over quick.
This is where it gets tricky. I could take that mapping data I got from IGDB to start with and try and do some automation there. Make sure the names match exactly, platforms have some kind of overlap, release date is within a certain timeframe, etc. I’d still want to go through any game over a certain number of shelf entries and manually make sure. It probably stands to reason that if there is only one game in the Grouvee database that has a given name, and that IGDB only has one game in their database with a given name, it’s almost assuredly the same game. I’m sure I could use that logic with all the newer games that don’t have a mapping too.
I’m going to have to do something to this effect. I’m just surprised how many games have the same name that are actually different (like the games SameGame!) I agree that Main games are the only thing we should probably worry about mapping right now. The DLC will be another fun mess itself, although there isn’t a ton of DLC in the Giant Bomb database.
Sounds cool and quite promising.
when you say rating you mean like a 1 thru 5 star rating right? Technically I thought this was the same thing as a “Review” (whether you actually write anything about a game or not) Where you can already score different releases/versions this way. And then you can select any of these reviews as the “Main Review” Right? If you can still do that all the same way with the new database, then I dont see the problem with automatically rolling up the score to the main game page by default Because you could always go back in and fine tune it to specific releases. Minimum amount of initial clicks/work is probably better for more users.
and yeah the DLC getting its own pages was a neat idea, but it seems the giant bomb database was terribly inconsistent at this. If IGDB curates that already, major boon
Any news on this topic? I’m looking forward to having my game collection in IGDb. I understand it’s a very complex task to migrate the data, I’m wondering what’s the current status and if we have some kind of understanding how long it will take.
I’m kind of embarrassed how long it’s been since we’ve talked on this thread. I’ve actually got all the code written that downloads data from IGDB and handles all the classification stuff. All the links between versions of games are handled, chapters of games, etc. I’ve mostly just gotten stuck trying to figure out how to merge the datasets, and I’ve just stalled out on it. The last decision I made was that I was just going to download all the IGDB data, and slowly merge it in the background. I’m scared to death of trying to do the crossover and just scaring everyone off. I’ve got to get something going though. I could make a million excuses about how I’ve moved recently, but I’ve simply been procrastinating it. I tend to try and just ride waves of motivation when they come, and when I don’t feel motivated, just leave it be. I really hope to get working on it here in earnest again as it gets cold here in Indiana.
I hope that gives some insight! Feel free to fire away with anymore questions.
I was a little sceptical when I heard about the switch to igdb. But I’ll say, they seem to have literally everything in their DB. I have yet to find a single game that I was looking for that’s not listed on the site.
Admittedly, I didn’t try searching for super-obscure itch.io games, but at least on the commercial side they seem to be super up-to date with everything. So now I’m actually excited for not having to add a good half of all games I’m playing to the giantbomb api myself in the future.
In regards to verifying unclear matches - why not involve the community? I don’t necessarily mean actively telling people to match data entries (even though that could certainly be an option), but rather to give the community tools to report incorrectly matched games, e.g. by displaying current and proposed new data on the same page and having a button where users report entries that are not correctly matched?
To be fair, that would probably involve a good amount of additional work to implement that, and then some more to actually bring everything to the front end in an easy-to-understand way.
Thanks for the update. I get how motivation can come and go in waves. Happy to help in whatever way makes sense, even if it just means coming back to the site and engaging.
Have you considered adding a landing page or a link on the main Grouvee site explaining the thought process and all the work that has and will go into the shift? I know it took me a while to realize the forums were a thing, and I think having open insight into the creator’s thought process would be the best way to trust a switch like this if you are still worried about folks leaving.
I forgot to respond when this topic resurfaced, but I just wanted to say that I’m still looking forward to the switch, whenever it happens. Happy to help with some of the migratory drudge-work, when/if needed. I’ve started making some contributions to IGDB in preparation. I’ve looked around, and I really like how much more comprehensive/permissive their database is:
They link compilations to comprised games (e.g. Uncharted: Drake’s Fortune is listed as one of the games bundled in Uncharted: The Nathan Drake Collection – Giant Bomb doesn’t allow the latter as a release of the former)
Their definition of a ‘game’ seems broader, at least in practice (e.g. Michael Joyce’s afternoon, a story is in there)
Their range/definition of platforms also seems broader (e.g. DVD Player)
The only thing that really “”““scares””“” me, is having to re-check my entire shelves to see if I got the exact version of the game I played, since I know IGDB likes to have several versions of the same game as separate entries.
So, for example, I’d have to change Katamari Damacy to Katamari Damacy Reroll, or MGS3 to MGS3 HD edition.
And that’s probably going to change the release date of each game I added, kinda messing up the top-10 per year list I was planning on making.
But I also like to organize stuff so I’m probably going to have a good time doing it lmao.
I swear I’m still working on this when I get time. I’m maybe getting a kick in the pants here because I noticed that my update script from Giant Bomb hadn’t finished in the last week or so. It looks like they might have changed the rate limits in such a way that I’d have to redo my entire process. I’m waiting to hear back still, but if I’m going to update anything, it might as well be to go to IGDB. At the same time, Amazon/Twitch laid off some people over at IGDB recently. Video game data is impossible!
If it’s any help to you, the game coverage on giantbomb is getting worse as well. I started helping to update their wiki at the same time I started using grouvee, around 2015 - purely for self-serving purposes, to add games that are missing in my grouvee backlog.
Back then they already had most games I was looking for, and I only needed to add a few smaller ones. Over the years this has shifted a lot, and nowadays I have to add a good 35%-40% of my backlog mysfelf. At any given time I have a good dozen or so games that I need to add to their database and just procra… am too busy to do it.
I’ve entirely given up on VR games, because other than the biggest releases I basically need to add every game I want to play myself.
I’m not entirely sure if the number of people willing to contribute to giantbomb has gone down in absolute numbers, or just relative to the growth of the games industry in the last decade, but it’s definitely noticable that the coverage on the site is way lower than it was back then.
I’m in a similar boat. I play a lot of smallish indie games, and the coverage of those games here is very sparse. All of the games I’ve manually added recently have been on IGDB, though. There’s been a few games that I just haven’t bothered adding here lately since it gets a little frustrating not being able to enter developers and such that aren’t already in the database and having to put out a rather bare page.
Don’t get me wrong, though, I love Grouvee and appreciate all of peter’s hard work! Grouvee has just fallen behind some of the competition (e.g. backloggd) for my use case. The community is better here, so I’m not going anywhere, but it would be great to get on IGDB!
If IGDB curates that already, major boon