The NEW Build Your Own Arcade Controls

Main => Artwork => Topic started by: yo1dog on August 14, 2018, 09:14:05 am

Title: Arcade Artwork Vault
Post by: yo1dog on August 14, 2018, 09:14:05 am
TLDR; Lets build something to archive, consolidate, and serve arcade artwork that is reliable, easy, and cheap.

After coming back to the arcade scene after a few years, I have found that one of the best sources for vector art, Arcade Art Library, is gone and probably for good. As a community, we do so much to preserve various aspects of Arcade history. Why does it seem like artwork is left to float around on various unstable sites that go down without warning?

We need a digital vault. This vault could serve two purposes: A. maintain archives/backups and B. serve artwork to the public. Part A is the most important; We can always figure out how to distribute the content later.

Part A: The backups can be very reliably and cheaply maintained using Amazon Glacier. The cost for 1TB (1024GB) of storage is $4.10/mo. This would prevent the massive loss of assets that occur when sites like Arcade Art Library go down. We could ask for or write quick scripts to get (with permission) dumps of the current popular arcade art sites and throw them in the vault. I am not sure how often new artwork is uploaded to these sites, but we would have to figure out how to keep the backups updated. However, I think just the initial dump is very useful. Also, this may not be a problem if we accomplish all of part B.

Part B: Serving the artwork dumps to the public can be accomplished at several levels. First, the artwork would need to be stored somewhere that can be accessed readily (Amazon Glacier is only for long term storage and not real-time retrieval). Amazon S3 (Simple Storage Service) is a good choice. Again it is very reliable and cheep. 1TB of storage with 10,000 requests and 100GB of traffic is $31.16/mo.

The first level would be to simply host a static directory file on S3 that lists all the assets in the vault with links. This an extremely simple and reliable way to provide the public access to all the files in the vault. Users could just search the page in their browser for whatever they are looking for.

The second level would be a simple, static, JavaScript enabled page(s) (again hosted on S3) that provide users a minimal interface for finding and browsing artwork from the vault.

The third and final level would be a fully featured website that also allows users to upload artwork. The server could be hosted using Amazon Beanstalk or could be built serverless using Amazon Lambda. The site could utilize services like Amazon CloudSearch/Elasticsearch for robust search features. Databases could be hosted by Amazon RDS/DynamoDB. Rating systems and curation tools could be implemented.

I think all three levels should be implemented in order and remain. This way they act as fallbacks. If the fancy website goes down years later with no one around to fix it, the basic website still works. If that fails for some reason as well, the static directory listing will always be available.

Further, if we can consolidate and organize the currently spread-out artwork, we can provide a valuable resource to the community. If we provide the tools to easily upload new artwork, we can maintain the consolidation while encouraging people to share their work. The rating system will help bubble up true/accurate recreations. I also have a few ideas on how to use ffmpeg/ImageMagick to automatically extract image metadata so that artwork could be filtered by type and resolution. A user could specify the print size and filter images > 300dpi.

Who's going to do it? I will. As a professional backend engineer of 10yrs, I have ample experience to get this built. Compared to the systems I build and maintain on a daily basis, this is actually pretty darn simple and easy. I might need some help getting a website too look pretty, though :P.

Who's going to pay for it? I will. I don't know the actual numbers, but it doesn't look like much. I most likely can't/won't run ads because of the nature of the content (can't make money distributing copyrighted content, I would assume). This community has given so much to me, I don't mind giving back!

Why is Amazon mentioned so much? Amazon Web Services (AWS) is a suite of cloud/virtual hardware services. They are one of the biggest players in the space, cheap, and I use them daily for my job so I am very familiar with them.

How can I help? If you run any of the popular arcade artwork sites, please contact me. I would live to get some details from you even if you don't wish to participate. Details on the number of files, the total file size, and the average monthly bandwidth usage would be very valuable for helping me estimate costs.

If you are a developer and want to help out, let me know. Also let me know of any opinions you have on the architecture.

If you know any arcade artwork libraries not included in the list posted here (,17346.msg1372720.html#msg1372720), let me know.

If you have any ideas or comments, let me know!

What do you guys think?

 - Mike
Title: Re: Arcade Artwork Vault
Post by: lilshawn on August 14, 2018, 01:11:01 pm,156600.msg1645393.html#msg1645393 (,156600.msg1645393.html#msg1645393)

Title: Re: Arcade Artwork Vault
Post by: yo1dog on August 14, 2018, 07:18:28 pm
Ah, didn't realized something similar was discussed recently. Thanks for the link.

After reading through the thread, it seems people are concerned about who can have access. That seems silly to me. Regardless, that concern does not apply to the most important part of this project: the vault. We need to consolidate the artwork in a safe place instead of it floating around on websites and people's hard drives. Once we accomplish that we can discuss if/how to distribute it.

If information about the author is attached to the artwork, we can setup an automatic best-effort attempt to verify if the artwork can be shared.