Oh wow I did not know that was happening at giantbomb.
I still contribute semi-regularly to GB, but can say that any data that needs moderator approval is definitely taking substantially longer to get the nod. I reported two duplicate game pages about a month ago and they still haven’t been addressed. 
4 Likes
You’re right. It’s not acceptable, and it’s my fault. I don’t have time to keep up with it, I haven’t built a good enough system to let the community do it, and I don’t have the resources to pay the volunteers who help me to do it. I owe such much to all the volunteers who have donated their time that have helped me over the years. I wish I could pay everyone a full time salary to do it.
I promise I’m working on the transition to IGDB when I have time. I know other sites like backloggd have better data, but I really do think we have a better shelf system. I just need to pair the better data with the better shelves and we’ll be back at full force again.
7 Likes
Here’s the thing: at the end of the day, Giant Bomb may not be perfect, but it is working. And it’s important for everybody to remember that this is a free resource that you are giving to us in your spare time. So please don’t feel that the discussion here is meant to be any kind of critique of your priorities, or pressure on you to give us…you know…a better free resource.
Personally, I think Grouvee’s a great site. I like the shelves, I love the community, I plan to be here a while and I’m excited for whatever changes come in their due time. Unless you plan to make the site tarantula-themed, in which case I take all of it back. 
8 Likes
+1 what @Jeslie wrote.
Don’t beat yourself up about taking your time, @peter. It sounds like the transition will require a lot of work, and no-one could reasonably expect you to do it quickly. (Also – more haste, less speed, and all that.)
6 Likes
Is this still happening? I wanted to add some random horror games to the database, but, if this change is happening somewhat soon, I might just wait and see if they are already in the new database
2 Likes
It is still happening. I’m so far behind in life, it’s not even funny, but it will still happen. I’d go ahead and submit them here and we’ll merge them together if they do happen to be on IGDB. That’s actually the main sticking point is figuring out how to do the merge in a sane way. Importing data and displaying it with all the new data points has been done for a long time. Figuring out how to associate their games with our games is my nightmare 
Maybe I’ll make a small horror game about it.
5 Likes
A horror game about coding? I like it. Id play it. Maybe the game begins coding itself. Or you get pulled into a game you were coding and you have to battle it via coding lol
4 Likes
Maybe this doesn’t look like much, but there’s a ton that’s gone on in the background to get this data included. This is a vague idea of what the pages can and will include. There’s a whole lot of linking going on in the database. I just need to figure out how to map our games to the IGDB games using community tools, and we’ll be on our way!
13 Likes
Ha, half of this excites me lol.
2 Likes
Haha. I need to know what the two halves are!?
2 Likes
One half is that youre making progress and including consoles, which is the exciting part.
The other half is youre adding more work to your already full plate lol. Which less exciting XD
7 Likes
I love how organized and digestible this information is. So many websites feel so cluttered with the way they handle data but this has me really excited for the update.
4 Likes
Also do we still have likke the comments and reviews after the merge? Are they like stashed away all nice n neat on anoyher page?
4 Likes
Yeah, all that stuff will still be there. Right now all I put on a game page is the 10 most hearted reviews, and the 10 latest statuses. I’m hoping to show that many posts, plus links to more. I don’t know why I’ve never built a page to show more reviews/statuses for a game. Right now, people’s reviews for big games just kind of get lost in the ether after they go off the front page. There’s still a fair amount more data to get on the page (release dates!), but like @Roach is saying, the less clutter the better.
Speaking of release dates. That is an extremely annoying and weird data point to handle. IGDB has a first_release_date field, which is basically what Grouvee stores now for a game. They have a whole other list of release dates I could grab from the database too. For instance, the original Legend of Zelda, the first release date is 2/20/1986, but then they list all the other release dates on different consoles in different regions. It’s kind of like our releases list now, but just for dates. I’ll probably just do what IGDB does (and what we’ve been doing), and use the first release date as the release date used for sorting and displaying, and then maybe do some kind of table of the other release dates for informational purposes. That’s honestly the only data point from them I haven’t written the import code for yet.
4 Likes
Hey, just getting caught up here as I somehow missed this entirely.
I’m not in a place right now to be responsible for organizing, but I’d love to help with grunt work. Anyone can feel free to reach out to me on here if you need a few hours of manual/digital labor or spreadsheet work.
3 Likes
So I just had a thought, and maybe it’s crazy. Maybe we don’t have to merge data at all?
What if we just downloaded the entire IGDB database, and then just made it so when you search for games on the site, only the IGDB games come up. Everyone’s shelves would stay the same, reviews, statuses, etc. with all the old entries from Giant Bomb. This way we don’t inadvertently destroy anyone’s shelf data. You’d still be able to look at old game entries, but you wouldn’t be able to shelve them or review them.
I’m trying to think of the main issues that would pop up from this approach.
There will be no reviews or statuses that show up on the new pages to start with, so the site might look dead to new users. I can remedy this with messaging and build tools to let people migrate reviews over to a new entry. Maybe in the case where a game is clearly an exact one to one match, we merge them.
What would be the pitfalls of this that I’m not thinking of?
1 Like
Im nit sure i fully understand this. But as long as i dont lose all my reviews and such im cool with it. Feels like any of it is going tk be a lot to do with some growing pains
1 Like
There are a few additional things to consider.
- One is aggregated data like user ratings or completion times will be incorrect because that data will be split between existing game entries and new ones.
- Another one is possible data drift. The longer two somewhat identical datasets exist, the more they will drift apart over time, making them progressively harder reconcile.
- Also, users will likely be confused as to why there are suddenly a bunch of duplicate games.
Overall I think it’s absolutely worth considering. For a lot of the issues there are possible migitations:
- Having clear new/legacy (or something along those lines) indicators to distinguish data from different providers
- Encouraging users to use the new igdb entries rather than the existing giantbomb ones, e.g. by making old data read only, or putting up banners
- Providing an easy method to link old and new entries like you mentioned
- Providing some an additional way for users to mass-migrate data, similar to the thing where you can link games on grouvee to steam games
- Using that crowd sourced data to gradually improve data quality. Like, if a few dozen people want to migrate data between the same source and target game, it might be safe to automatically merge them
- Making sure that users are informed in detail about the migration, both to keep the support load low(er) and to keep users happy because they know what’s going on.
- Recruiting temporary volunteers to either help clean up some of the data and/or help with user support
2 Likes
The way I was going to do this, and it’s the main reason I haven’t pulled the trigger yet, was that I was thinking that I was going to have to merge our existing game database with IGDB’s. Here’s an example. IGDB has an entry for the 64 version of Ocarina of Time, and we have a Giant Bomb entry for Ocarina of Time. All of the user data is associated with the Giant Bomb entry. When downloading the IGDB dataset, my code would create a new entry that has the IGDB data and IDs and links to the remakes, etc. I was initially thinking it was going to be a nightmare trying to match the games up and merge them together, but I don’t think we necessarily have to do that.
We’d still have the existing Giant Bomb entry in the database, but it wouldn’t show up in the search anymore. The IGDB entry would show up instead. If you added the GB entry to your shelves, or written a review, etc. all of that would still exist exactly as it was. But if you wanted to do anything new, you’d have to use the IGDB data. I would have tools to move your data to a different game as well.
Hopefully that makes sense!
2 Likes