Settlement Generation Challenge 2020

DEPRECATED: This is the overview of the past 2020 GDMC Challenge - archieved for historical reference. Please refer to the current competition page for rules, submission options, etc. The Settlement Generation Challenge is about writing an algorithm that can create a settlement for a given, unknown Minecraft map. The challenge is to produce an algorithm that is adaptive towards the provided map, creates a settlement that satisfies a range of functional requirement - but also looks good and evokes an interesting narrative. The goal is to basically produce an algorithm that can rival the state of the art of what humans can produce. So far there have been 2 iterations of the challenge: once in 2018, and again in 2019. To participate, you have to write a program that can build a settlement for an unknown Minecraft map. After you submit your algorithm, we will run it on 3 previously unseen maps, and a panel of human judges will then evaluate the resulting settlement, based on a set of criteria focusses on Adaptation to the Environment and Terrain, Functionality from an Embodied Perspective, Narrative Integration and Visual Aesthetics.

Years Active 2018, 2019 Conferences FDG, ICCC Submission Languages Java, Python Submission Types Forge Mod, MCEdit Filter, Schematic Manipulator

Rules Team submissions are allowed. One submission per person / team only. For compatibility reasons, we will work with Minecraft Version 1.12. We will update all outlets if this changes. Winners will be announced during an international artificial intelligence and gaming conference (COG, FDG, ICC, etc). Which conference will be announced well beforehand. At the same time we will also publish the detailed evaluation results online for each algorithm. Once the results have been announced we will also publish the competition maps, the settlements generated by the algorithms, and the participants description of their algorithm. We also encourage participants to publish their code, but this will not be mandatory. If participants agree to share their code, we will make it available on our website with an appropriate open source license. If they decline, we will not publish the code. In either case, the organizers will not take exclusive ownership of the code. Evaluation The submitted algorithms will be judged based on a set of 3, different, unknown maps. Example training maps will be provided. The competition maps will not be publicly available until after the competition. Every submitted algorithm will be run on the same 3 unknown maps, and the resulting maps with settlements will be anonymized and given to the judges. The judges will also be able to see the 3 maps in their natural state, before the algorithm was applied. The algorithms should not run substantially longer than 10 minutes on a regular PC for a 256 by 256 map, and not exceed 2 Gbit of RAM. Algorithms will be disqualified if they run considerably longer than 10 minutes on a single map. The competition maps will be around 256 by 256 in size, but the exact size of the competition maps will vary. The algorithm will be given the exact dimensions of the map when it is run and should be able to adapt to slightly bigger or smaller maps. For example, a competition map might be 200 by 300 in size. The competition is judged by a set of human judges who are either member of the advisory board of delegates selected by them. If a member of the advisory board or the organization team ends up submitting an algorithm themselves, they will be excluded from the evaluation process for that year, i.e. they will not be allowed to nominate judges or judge themselves. The plan is to have all judges evaluate all algorithms and average their scores. If this is not possible, due to high number of submissions, there might be several evaluation rounds, where earlier rounds are used to reduce the number of submissions by evaluating a lower number of maps with only a subsection of the judges. The judges will then evaluate all 3 maps for a given algorithm by spawning at the central point (for the x and z coordinate) of the map. While the judges are in creative mode, they are encouraged to walk through the settlement from a survival perspective. They are given a set of evaluation criteria (see Evaluation Criteria) and are asked to assign points based on the performance of the algorithm across all maps. The sum of the points is the score of the algorithm. The algorithm with the highest score wins. Evaluation Criteria We defined a set of evaluation criteria based on what we believe the challenges in moving towards more human-like settlement generation are. So, when evaluation the generated settlements judges are asked to score each generator in four categories: Adaptability, Functionality, Narrative and Aesthetics. Each category is worth 10 points, and the following questions are provided to the judges to guide their evaluation. These questions are illustrative of the category, but not exhaustive. They should provide judges with an idea of what this category is about, but do not imply that they cover the complete scope the category. Adaptability Functionality Narrative Aesthetics Do the structures in the settlement adapt to the terrain? Does the settlement provide protection from danger? Does it keep mobs out/from spawning? Protection from other environmental dangers? Is the settlement evoking an interesting story? could you give a short description of what this settlement is about that sets it apart from other settlements? Does the settlement look good? Do the structure in the environment reflect the environment, i.e. usage of available material, adaptation to the biome? Is the settlement accessible to a player avatar in survival mode? Can you walk to everywhere? Is it easy? Faster modes of transport? How easy is it to find your way around? Is it clear what the function of the settlement is? Does this function make sense in regards to the terrain and environment it is in? I.e. is the logging camp in a forest, the harbour town at the sea? Is there a consistent look to the settlement? Does it appear that all structures belong to the same settlement? Does the settlement take advantage of terrain features or compensate for problems with the terrain? Does the settlement reflect the embodiment of the player avatar? Is it appropriately scaled? Is the functionality of the settlement supporting this narrative function? I.e. does the fortified frontier settlement have functioning walls, is the farming village equipped with functioning fields? Is there an appropriate level of variation in the existing structures? Are the settlements different in reaction to the different initial maps? Does the settlement makes ressources easy to obtain? Is it easy to get food? Does the settlement provide the player with additional affordances? Does the final settlement give any indication of how the settlement developed? Can you look at the settlement and imagine the order things were built, or what stages the development of the settlement took? Is there an indication of the history of the settlement evident in the structure? Are there any jarring features that make the settlement look unbelievable? Are there any other ways in how the settlement adapt to the given maps? Does the settlement provide functionality to the villagers? Are the any convincing and consistent allusion to human cultures or specific points in history that the settlement is modelled after? Does the settlement have its own culture? Can you tell things about this culture just by observing the settlement? Chronicle Challenge The 2nd iteration of the competition introduced the chronicle challenge. This was an option extra challenge, that was to be evaluated separately from the regular competition. Contestants were to indicate that their entry was competing in this challenge during submission. For the chronicle challenge, the generator produces a written book in Minecraft and places it inside the generated settlement – ideally in a place where it is easily found. The book should contain the chronicle of the settlement (in English) and should contain a narrative on how this particular settlement came about. The entries are evaluated based on how well the given chronicle fits the generated settlement and on overall quality. Submission See valid submission methods page to see all current methods. Challenges This section is about the specific challenges in generating a Minecraft settlement on a given map. We believe that these are some of the core obstacles that need to be overcome to move closer to human-level design. We should point out, though, that this list is not exhaustive. It is likely that there are other challenges and obstacles to get to human-like settlement generation. Furthermore, we consider the challenges outlined here ambitious goals to move towards, and do not expect them to be fully solved right away. The evaluation criteria that will be given to the judges are based on these challenges, because we want participants to address these problems. Adaptation to Environment and Terrain Real-life human settlements are adapted towards their surrounding environment and terrain in multiple ways. On a per-building basis, houses and structures reflect the climate conditions they are built in. Furthermore, the environment of a settlement might also dictate the available resources for construction, and certain conditions might require specific types of buildings, such as irrigation, shelter, etc. A house in the cold climate of Edinburgh, for example, might be built sturdy, to protect against wind and cold and keep the warmth in. Likewise, the easy availability of stone might influence the choice of building material. Real-life settlements are also adapted to the immediate terrain they are in. Natural features, such as mountains or lakes, play a role in the selection of where to settle. Some terrain features can be both positive and negative for a specific settlement, and settlements are often built to take advantage of the benefits and to compensate for the problems. A river, for example, might provide more food and added mobility via ship travel, but might also limit mobility by foot, and be a potential flooding hazard. A settlement might subsequently be built close to the river (or even more likely, at a good location to cross the river), to take advantage of the benefits. The same settlement might also build a dam to protect against flooding, and houses close to the water might sit on stilts. If the settlement dumps waste into the river there might also be a development of different districts, where better, more desirable houses are further up on the river. On the other hand, humans change the terrain and environment in a limited fashion to better suit their needs. While it might be impossible to remove a mountain or an ocean, it might be possible to build a canal or tunnel. In summary, human settlements are shaped by and in turn shape the terrain and environment around them. If we study human-built settlements in Minecraft, we can see that these principles are realized in some existing builds. AI settlement generators, on the other hand, struggle to adapt to environment and terrain. They often rely on flattening large areas and then building settlements that do not reflect the surrounding map at all. Similarly, the houses and structures built by them are often templates that get placed as a whole unit, and are in no way reflective of the environment they are in. As part of this challenge, we encourage participants to develop generators that produce different settlements for different kinds of maps, reflecting the available materials, surrounding terrain and environmental conditions. In particular, successful entries should generate settlement that fit into the topography of the map without changing it too much, and build structures that fit into the topography. We also encourage participants of the challenge to think of other ways in which the given maps influences the generated settlement, ideally in such a fashion that it is evident to a human observer what kind of adaptation happened. Ideally, different maps should lead to different settlements, and it should be clear in which way the settlements where shaped by the map. Also note, that there is some overlap with the other three criteria, as their particular solutions are also often ideally adapted to the underlying map. Functionality from an Embodied Perspective Minecraft is not just a creativity tool, but also a game. Subsequently, a Minecraft settlement is not just an aesthetic artifact; it also needs to fulfill certain functional roles. One big aspect of this is accessibility and mobility. Is it actually possible to walk through the settlement and reach its different parts? Are there doors into the buildings, are the bridges over the river? How easy is it to navigate the settlement? A second big function a settlement provides is protection from various dangers. Is the settlement well lit, or does it otherwise prohibit mobs from spawning? Does the settlement manage to keep dangerous mobs outside? Are there other forms of defenses and traps to protect the player? Finally, there is also a need to obtain food and process it - so is there an accessible way to produce food and not starve? This list of functional requirements is not exhaustive - it is just meant to give some examples. This criterion does have a certain overlap with the challenge of adaptivity. Certain functionality only needs to be provided in some cases. For example, bridges compensate for reduced mobility introduced by obstacles. They are an adaptation to reduce the negative effects of the terrain. Other functionality takes advantage of given terrain features, such as river lock further enhancing the mobility of an existing waterway. This should also be considered, not just from a player perspective, but also under the assumption that the NPC villagers living in the settlement have similar needs. There is crossover here with the next section, because some functionality displayed in typical human-build settlement is not functional in the strict sense. For example, in reality a lot of houses have pitched roof to ensure better rain water runoff. In Minecraft, this is not necessary (as rain does not pool on the ground), but players still build houses with pitched roofs. This narrative functionality alludes to a functionality we understand, yet is not strictly functional in Minecraft. We still encourage the inclusion of such functional elements. Ideally, these narrative functionalities should also be adaptive to the environment. So, pitched roofs should be more popular in a rainy swamp than in a dry dessert, etc. In summary, the fact that there is an actual player in the world gives us an embodied perspective. The avatar can interact with the world in a variety of different ways, and a settlement can provide different affordances to the player. The challenge in this competition is to produce a generator that can ensure that the settlement provides a maximum of functionality and affordances for the player, while also satisfying the other criteria. Believable and Evocative Narrative Human settlements are not just collections of functional buildings, but they also tell a story about how they came about, who are the people living there, and how they see the world. When we look at human-created Minecraft settlements, we can often see how certain human or imagined cultures are reflected in the created buildings. There are cities that resemble ancient Rome, mythical Elven forest outposts, and modern US cities. Often, the buildings also reflect very clear narrative ideas, such as “this is a defensive mining outpost built in a harsh environment”, or “this is a capital city, built to impress foreign and domestic visitors alike”. These settlements are evocative, i.e. they manage to transport this story by looks alone. Relating to the earlier challenge of adaptivity, the better settlements also evoke narratives that work well with the terrain and biome they are in. The narrative they evoke should fit the terrain and environment of the given map, and ideally arise from it. Different maps should result in different narratives. Human settlements also reflect how they came about. Medieval cores of modern cities, for example, tell a part of a settlement's origin story. This is something rarely seen, even in human-made Minecraft builds, as settlements here are created with the final state in mind. But procedural content generation does have a tradition of simulationist approaches, which would be a possibility for this challenge. An AI could simulate people living and building a settlement over several stages, rebuilding or replacing structures, or slowly modernizing them. This could solve several of the previous problems, as buildings and the overall settlement would be the actual result of an adaptive process, and the produce of an actual sequence of events forming the basis for a possible narrative. There is also a certain overlap between narrative and functional requirements. How easy the player can obtain wood for building and further process it is a functional requirement. To satisfy this a settlement could have a nearby forest and a workbench nearby. But to make this work with a narrative of a settlement that relies on wood production it might also be good to build a structure that looks like a sawmill, and a street to transport the goods to town. This would not produce any additional functionality to the player (or the digital villagers), but it would tell a narrative that aligns with the functionality provided to the player. Similarly, there could also be elements that have purely narrative functionality, i.e. structures that have a believable fictional use that is not immediately reflected in the game mechanics. For example, a settlement could have an aqueduct to provide water, even though Minecraft characters do not need to drink. To summarize, the particular challenge here is not just about narrative generation, but also about generating a narrative that fits with the generated settlement, and ensuring that this narrative is communicated to the player with the generated structures. Inspiration by existing, historical or imagined cultures can help to transport this narrative. The aim is to produce something that is both believable and evocative, while still being aligned with the previous criteria. Visual Aesthetics Aesthetics are arguably subjective, yet architects and city planners usually follow a range of principles when it comes to designing a settlement or a house. While untrained humans manage to intuitively realize these principles with some success, it is difficult to design an algorithm with automated aesthetic judgment. Existing Minecraft mods that add structures to the world circumvent this problem by hand-designing appropriate templates, and then building a settlement out of those templates. This solution has several problems: it allows for little variation between the buildings, as they all need to be pre-designed, and it also allows for little adaptation of the buildings to the surrounding buildings, the underlying terrain, etc. Buildings are sometimes parameterized, and hence reflect some specific environmental settings, such as available materials or the climate they are in. This makes them more adaptive, but there seems to be trade-off between controlling the exact look of a structure, and making it adaptable towards uncontrolled environmental factors. The challenge in our case is to ensure that buildings still follow basic design principles while being adaptive, functional and evocative. There is also the further problem that the overall settlement itself should follow certain aesthetic rules. How buildings are aligned and what sight lines exist can play an important role for the overall feel of the settlement. Finally, there is also the question of how well the particular aesthetic expression chosen aligns with the other challenges. For example, a settlement which has a narrative of being a foreboding fortress, and is designed with that functionality, should also have an aesthetic that reflects this. Bonus Challenge: Chronicle While cities implicitly tell a story about their inhabitants, cities also often have explicit stories about their history. These chronicles contain notable events that affected and shaped the settlement, such as fires, wars of periods of vibrant trade. The can also provide a lot of additional context, such as what was the function or reason for the settlement during different time periods, or what ideas and philosophies have shaped the settlements development. They also often contain stories about important people that shaped the city. Basically, anything that could be written about a city and would be achieved in the city is appropriate. For this challenge, your task is to create a chronicle that is currently stored in the city. It should be written as a Minecraft book, and should contain text relating to the generated city. This challenge relates back to several of our earlier criteria, such as adaptivity and narrative. The chronicle should be adapted to the actual generated city, and should the story of that specific city. The less generic this story is, the better. The chronicle challenge is optional, and entries will be evaluated based on the overall quality of the chronicle, and how well it fits with the different cities generated. Examples Here is a list of videos of good, human-created settlements that showcase some of the things we are looking for in a generator. Several Settlement builds by FyreUK Lakeside City Good examples for coherent theming of the overall settlement, adaptive towards its bay location (looks like a harbour city), houses well adapted to the height levels around the bay. The Land of Arkane Good example for theming, believable structures on the water, good integration of the existing height map. Good bridges and walkways to ensure mobility. Waterfall City Visually distinct districts, combination of singular large structures with smaller, more repetitive houses. The smaller houses are themed according to district, yet show a level of variability. Frozen Summit The building material and style of building reflects the environmental conditions. Also, larger structures that are adapted to the changing height. Other people: Monteriggioni Recreation Recreation of the historical city of Monteriggioni in Minecraft. Broville An example of a modern city in Minecraft. Also features some examples of interior design. FAQ I don’t like python, can I submit in Java? Yes, we offer an option to submit a Java based mod and a non-language-bound schematic manipulation submission option. For details, have a look at the submission guidelines. How much can I change the map? There are no limitations in the rules about how much you can change the map. You can add and remove as many blocks as you want. However, one of the evaluation criteria is adaptation to the terrain and environment. So, if you simply flatten the given map, and then build a settlement, you will not score high on adaptation. But there should be no problem with believable modifications to the terrain, especially if it is clear how they help the settlement. So, making room for important roads, filling in minor holes, digging a canal or tunnel to connect to part of the map, all these things should be fine. Removing a whole mountain might be a bit much, though. We might introduce a limitation on how many blocks can be changed for future competitions, but there is no limit for the first competition. Am I limited by the resources on the map? No, there is not rule that the settlement must be built from resources found on the map. On the other hand, predominantly building from resources that are present on the map is a form of adaptation to the environment. Then again, rich parts of a city might deliberately use resources that are hard to come by locally. What will the competition maps look like? The competition maps will be based what the build-in terrain generation in Minecraft produces. We will ensure that there are no existing surface settlements on the competition maps. There is no limitation in terms of biomes, so everything that you can find in a Minecraft world might theoretically pop up. We might slightly alter the maps to provide some additional terrain features or obstacles. The test maps should give you a rough idea of what you can expect. We aim to provide maps with a range of difficulty, so some maps should be easier to deal with than others. What will be the size of the maps? When running the algorithms, they will be provided with a bounding box – the area within the bounding box is the area in which the settlement needs to be build. The area will be around 256 by 256 blocks (and from bedrock to the sky limit) in size, but the exact size might vary. The algorithms should be able to adapt to varying map sizes. For example, they should be able to deal with a 200 by 300 blocks map. We guarantee that the maps are rectangular. Why are the maps only 256 by 256? There are two reasons for not making the maps too small. First, judges need to be able to evaluate a settlement in a reasonable time frame and should therefore be able to walk through and see most of the settlement within a few minutes. We also wanted to ensure that we can run all algorithms on our end in a reasonable time frame for the evaluation. Nevertheless, we encourage participants to write algorithms that can adapt to larger and smaller maps. Will I be disqualified if my algorithm runs longer than 10 minutes? The 10 minutes are a rough guideline – to give participants a general idea of how long they can take. If your algorithm takes a bit longer on our PC, then this is not a problem. Once it goes on for an hour or so, we will likely interrupt it. Will I be disqualified if my code does not run on your machine? If we cannot get your algorithm to run on our machine, then we cannot evaluate it performance. If you are worried, we suggest you submit early, and contact us. We are willing to have a look and see if your algorithm runs on our setup. You can replace your submission later with a new submission to update it. You talk about culture in your evaluation guidelines, can I build a fantasy city? Yes, our evaluation criteria are not limited to human or historical cultures or settlements. Keep in mind that there are two criteria, functionality and narrative. For functionality, you have to work with the game mechanics provided by Minecraft. So, your floating city should still have a way to get around for its inhabitants and should not be lethal to a player. But in terms of the narrative evocation, you are free to generate fictional of futuristic settlement – provided that they follow consistent rules, which should ideally be evident from the way the settlement is build.