GDC Vault is part of the Informa Tech Division of Informa PLC
This is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.
Virtual Sync: Terabytes on Demand
For decades, the basic way we work on games has remained mostly unchanged - we sync the game's source data from something like Perforce, we change a few things, we validate locally, and we submit our changes. With AAA game source data exceeding 10 TB, and content creators needing to work across multiple branches, it has become steadily less practical to require these volumes of data transfer. When the Covid-19 pandemic sent all of our employees to work from their homes, our development process ran into the limitations of multi-day network data transfers and we were faced with the need to physically transport hard drives back and forth. VirtualSync is our answer: a virtual file system that seamlessly fetches assets on-demand as-needed, allowing us to shrink up-front syncs by 98% or more. We show how this simple idea requires a number of complex engineering challenges to be solved. We will present numerous problems encountered and the solutions we developed to solve performance issues, operating system limitations, service monitoring at scale and more.
Did you know free users get access to 30% of content from the last 2 years?