DEPRECATED: This is the overview of the past 2020 GDMC Challenge - archieved for historical reference. Please refer to the current competition page for rules, submission options, etc. The Settlement Generation Challenge is about writing an algorithm that can create a settlement for a given, unknown Minecraft map. The challenge is to produce an algorithm that is adaptive towards the provided map, creates a settlement that satisfies a range of functional requirement - but also looks good and evokes an interesting narrative. The goal is to basically produce an algorithm that can rival the state of the art of what humans can produce. So far there have been 2 iterations of the challenge: once in 2018, and again in 2019. To participate, you have to write a program that can build a settlement for an unknown Minecraft map. After you submit your algorithm, we will run it on 3 previously unseen maps, and a panel of human judges will then evaluate the resulting settlement, based on a set of criteria focusses on Adaptation to the Environment and Terrain, Functionality from an Embodied Perspective, Narrative Integration and Visual Aesthetics.
|
|
Rules
Winners will be announced during an international artificial intelligence and gaming conference (COG, FDG, ICC, etc). Which conference will be announced well beforehand. At the same time we will also publish the detailed evaluation results online for each algorithm. Once the results have been announced we will also publish the competition maps, the settlements generated by the algorithms, and the participants description of their algorithm. We also encourage participants to publish their code, but this will not be mandatory. If participants agree to share their code, we will make it available on our website with an appropriate open source license. If they decline, we will not publish the code. In either case, the organizers will not take exclusive ownership of the code. EvaluationThe submitted algorithms will be judged based on a set of 3, different, unknown maps. Example training maps will be provided. The competition maps will not be publicly available until after the competition. Every submitted algorithm will be run on the same 3 unknown maps, and the resulting maps with settlements will be anonymized and given to the judges. The judges will also be able to see the 3 maps in their natural state, before the algorithm was applied. The algorithms should not run substantially longer than 10 minutes on a regular PC for a 256 by 256 map, and not exceed 2 Gbit of RAM. Algorithms will be disqualified if they run considerably longer than 10 minutes on a single map. The competition maps will be around 256 by 256 in size, but the exact size of the competition maps will vary. The algorithm will be given the exact dimensions of the map when it is run and should be able to adapt to slightly bigger or smaller maps. For example, a competition map might be 200 by 300 in size. The competition is judged by a set of human judges who are either member of the advisory board of delegates selected by them. If a member of the advisory board or the organization team ends up submitting an algorithm themselves, they will be excluded from the evaluation process for that year, i.e. they will not be allowed to nominate judges or judge themselves. The plan is to have all judges evaluate all algorithms and average their scores. If this is not possible, due to high number of submissions, there might be several evaluation rounds, where earlier rounds are used to reduce the number of submissions by evaluating a lower number of maps with only a subsection of the judges. The judges will then evaluate all 3 maps for a given algorithm by spawning at the central point (for the x and z coordinate) of the map. While the judges are in creative mode, they are encouraged to walk through the settlement from a survival perspective. They are given a set of evaluation criteria (see Evaluation Criteria) and are asked to assign points based on the performance of the algorithm across all maps. The sum of the points is the score of the algorithm. The algorithm with the highest score wins. Evaluation CriteriaWe defined a set of evaluation criteria based on what we believe the challenges in moving towards more human-like settlement generation are. So, when evaluation the generated settlements judges are asked to score each generator in four categories: Adaptability, Functionality, Narrative and Aesthetics. Each category is worth 10 points, and the following questions are provided to the judges to guide their evaluation. These questions are illustrative of the category, but not exhaustive. They should provide judges with an idea of what this category is about, but do not imply that they cover the complete scope the category.
Chronicle ChallengeThe 2nd iteration of the competition introduced the chronicle challenge. This was an option extra challenge, that was to be evaluated separately from the regular competition. Contestants were to indicate that their entry was competing in this challenge during submission. SubmissionSee valid submission methods page to see all current methods. ChallengesThis section is about the specific challenges in generating a Minecraft settlement on a given map. We believe that these are some of the core obstacles that need to be overcome to move closer to human-level design. We should point out, though, that this list is not exhaustive. It is likely that there are other challenges and obstacles to get to human-like settlement generation. Furthermore, we consider the challenges outlined here ambitious goals to move towards, and do not expect them to be fully solved right away. Adaptation to Environment and TerrainReal-life human settlements are adapted towards their surrounding environment and terrain in multiple ways. On a per-building basis, houses and structures reflect the climate conditions they are built in. Furthermore, the environment of a settlement might also dictate the available resources for construction, and certain conditions might require specific types of buildings, such as irrigation, shelter, etc. A house in the cold climate of Edinburgh, for example, might be built sturdy, to protect against wind and cold and keep the warmth in. Likewise, the easy availability of stone might influence the choice of building material. Real-life settlements are also adapted to the immediate terrain they are in. Natural features, such as mountains or lakes, play a role in the selection of where to settle. Some terrain features can be both positive and negative for a specific settlement, and settlements are often built to take advantage of the benefits and to compensate for the problems. A river, for example, might provide more food and added mobility via ship travel, but might also limit mobility by foot, and be a potential flooding hazard. A settlement might subsequently be built close to the river (or even more likely, at a good location to cross the river), to take advantage of the benefits. The same settlement might also build a dam to protect against flooding, and houses close to the water might sit on stilts. If the settlement dumps waste into the river there might also be a development of different districts, where better, more desirable houses are further up on the river. On the other hand, humans change the terrain and environment in a limited fashion to better suit their needs. While it might be impossible to remove a mountain or an ocean, it might be possible to build a canal or tunnel. In summary, human settlements are shaped by and in turn shape the terrain and environment around them. If we study human-built settlements in Minecraft, we can see that these principles are realized in some existing builds. AI settlement generators, on the other hand, struggle to adapt to environment and terrain. They often rely on flattening large areas and then building settlements that do not reflect the surrounding map at all. Similarly, the houses and structures built by them are often templates that get placed as a whole unit, and are in no way reflective of the environment they are in. As part of this challenge, we encourage participants to develop generators that produce different settlements for different kinds of maps, reflecting the available materials, surrounding terrain and environmental conditions. In particular, successful entries should generate settlement that fit into the topography of the map without changing it too much, and build structures that fit into the topography. We also encourage participants of the challenge to think of other ways in which the given maps influences the generated settlement, ideally in such a fashion that it is evident to a human observer what kind of adaptation happened. Ideally, different maps should lead to different settlements, and it should be clear in which way the settlements where shaped by the map. Also note, that there is some overlap with the other three criteria, as their particular solutions are also often ideally adapted to the underlying map. Functionality from an Embodied PerspectiveMinecraft is not just a creativity tool, but also a game. Subsequently, a Minecraft settlement is not just an aesthetic artifact; it also needs to fulfill certain functional roles. One big aspect of this is accessibility and mobility. Is it actually possible to walk through the settlement and reach its different parts? Are there doors into the buildings, are the bridges over the river? How easy is it to navigate the settlement? A second big function a settlement provides is protection from various dangers. Is the settlement well lit, or does it otherwise prohibit mobs from spawning? Does the settlement manage to keep dangerous mobs outside? Are there other forms of defenses and traps to protect the player? Finally, there is also a need to obtain food and process it - so is there an accessible way to produce food and not starve? This list of functional requirements is not exhaustive - it is just meant to give some examples. This criterion does have a certain overlap with the challenge of adaptivity. Certain functionality only needs to be provided in some cases. For example, bridges compensate for reduced mobility introduced by obstacles. They are an adaptation to reduce the negative effects of the terrain. Other functionality takes advantage of given terrain features, such as river lock further enhancing the mobility of an existing waterway. This should also be considered, not just from a player perspective, but also under the assumption that the NPC villagers living in the settlement have similar needs. There is crossover here with the next section, because some functionality displayed in typical human-build settlement is not functional in the strict sense. For example, in reality a lot of houses have pitched roof to ensure better rain water runoff. In Minecraft, this is not necessary (as rain does not pool on the ground), but players still build houses with pitched roofs. This narrative functionality alludes to a functionality we understand, yet is not strictly functional in Minecraft. We still encourage the inclusion of such functional elements. Ideally, these narrative functionalities should also be adaptive to the environment. So, pitched roofs should be more popular in a rainy swamp than in a dry dessert, etc. In summary, the fact that there is an actual player in the world gives us an embodied perspective. The avatar can interact with the world in a variety of different ways, and a settlement can provide different affordances to the player. The challenge in this competition is to produce a generator that can ensure that the settlement provides a maximum of functionality and affordances for the player, while also satisfying the other criteria. Believable and Evocative NarrativeHuman settlements are not just collections of functional buildings, but they also tell a story about how they came about, who are the people living there, and how they see the world. When we look at human-created Minecraft settlements, we can often see how certain human or imagined cultures are reflected in the created buildings. There are cities that resemble ancient Rome, mythical Elven forest outposts, and modern US cities. Often, the buildings also reflect very clear narrative ideas, such as “this is a defensive mining outpost built in a harsh environment”, or “this is a capital city, built to impress foreign and domestic visitors alike”. These settlements are evocative, i.e. they manage to transport this story by looks alone. Relating to the earlier challenge of adaptivity, the better settlements also evoke narratives that work well with the terrain and biome they are in. The narrative they evoke should fit the terrain and environment of the given map, and ideally arise from it. Different maps should result in different narratives. Human settlements also reflect how they came about. Medieval cores of modern cities, for example, tell a part of a settlement's origin story. This is something rarely seen, even in human-made Minecraft builds, as settlements here are created with the final state in mind. But procedural content generation does have a tradition of simulationist approaches, which would be a possibility for this challenge. An AI could simulate people living and building a settlement over several stages, rebuilding or replacing structures, or slowly modernizing them. This could solve several of the previous problems, as buildings and the overall settlement would be the actual result of an adaptive process, and the produce of an actual sequence of events forming the basis for a possible narrative. There is also a certain overlap between narrative and functional requirements. How easy the player can obtain wood for building and further process it is a functional requirement. To satisfy this a settlement could have a nearby forest and a workbench nearby. But to make this work with a narrative of a settlement that relies on wood production it might also be good to build a structure that looks like a sawmill, and a street to transport the goods to town. This would not produce any additional functionality to the player (or the digital villagers), but it would tell a narrative that aligns with the functionality provided to the player. Similarly, there could also be elements that have purely narrative functionality, i.e. structures that have a believable fictional use that is not immediately reflected in the game mechanics. For example, a settlement could have an aqueduct to provide water, even though Minecraft characters do not need to drink. To summarize, the particular challenge here is not just about narrative generation, but also about generating a narrative that fits with the generated settlement, and ensuring that this narrative is communicated to the player with the generated structures. Inspiration by existing, historical or imagined cultures can help to transport this narrative. The aim is to produce something that is both believable and evocative, while still being aligned with the previous criteria. Visual AestheticsAesthetics are arguably subjective, yet architects and city planners usually follow a range of principles when it comes to designing a settlement or a house. While untrained humans manage to intuitively realize these principles with some success, it is difficult to design an algorithm with automated aesthetic judgment. Existing Minecraft mods that add structures to the world circumvent this problem by hand-designing appropriate templates, and then building a settlement out of those templates. This solution has several problems: it allows for little variation between the buildings, as they all need to be pre-designed, and it also allows for little adaptation of the buildings to the surrounding buildings, the underlying terrain, etc. Buildings are sometimes parameterized, and hence reflect some specific environmental settings, such as available materials or the climate they are in. This makes them more adaptive, but there seems to be trade-off between controlling the exact look of a structure, and making it adaptable towards uncontrolled environmental factors. The challenge in our case is to ensure that buildings still follow basic design principles while being adaptive, functional and evocative. There is also the further problem that the overall settlement itself should follow certain aesthetic rules. How buildings are aligned and what sight lines exist can play an important role for the overall feel of the settlement. Finally, there is also the question of how well the particular aesthetic expression chosen aligns with the other challenges. For example, a settlement which has a narrative of being a foreboding fortress, and is designed with that functionality, should also have an aesthetic that reflects this. Bonus Challenge: ChronicleWhile cities implicitly tell a story about their inhabitants, cities also often have explicit stories about their history. These chronicles contain notable events that affected and shaped the settlement, such as fires, wars of periods of vibrant trade. The can also provide a lot of additional context, such as what was the function or reason for the settlement during different time periods, or what ideas and philosophies have shaped the settlements development. They also often contain stories about important people that shaped the city. Basically, anything that could be written about a city and would be achieved in the city is appropriate. For this challenge, your task is to create a chronicle that is currently stored in the city. It should be written as a Minecraft book, and should contain text relating to the generated city. This challenge relates back to several of our earlier criteria, such as adaptivity and narrative. The chronicle should be adapted to the actual generated city, and should the story of that specific city. The less generic this story is, the better. The chronicle challenge is optional, and entries will be evaluated based on the overall quality of the chronicle, and how well it fits with the different cities generated. ExamplesHere is a list of videos of good, human-created settlements that showcase some of the things we are looking for in a generator. Several Settlement builds by FyreUKLakeside City The Land of Arkane Waterfall City Frozen Summit Other people:Monteriggioni Recreation Broville FAQI don’t like python, can I submit in Java? How much can I change the map? Am I limited by the resources on the map? What will the competition maps look like? What will be the size of the maps? Why are the maps only 256 by 256? Will I be disqualified if my algorithm runs longer than 10 minutes? Will I be disqualified if my code does not run on your machine? You talk about culture in your evaluation guidelines, can I build a fantasy city? |