|
This is the overview page for the fourth iteration of the GDMC AI Settlement Generation Challenge in Minecraft, running in 2021. This competition is in the past, but this page exists to document the rules and outcomes of this past competition. Submissions for 2021 have closed on the 31st of May, 2021. The Settlement Generation Challenge is about writing an algorithm that can create a settlement for a given, unknown Minecraft map. The challenge is to produce an algorithm that is adaptive towards the provided map, creates a settlement that satisfies a range of functional requirement - but also looks good and evokes an interesting narrative. The goal is to basically produce an algorithm that can rival the state of the art of what humans can produce. So far there have been 3 iterations of the challenge: once in 2018, and in 2019, and in 2020. To participate, you have to write a program that can build a settlement for an unknown Minecraft map. After you submit your algorithm, we will run it on 3 previously unseen maps, and a panel of human judges will then evaluate the resulting settlement, based on a set of criteria focusses on Adaptation to the Environment and Terrain, Functionality from an Embodied Perspective, Narrative Integration, and Visual Aesthetics. Results Video
|
|
OverviewTo participate, you, or a team, have to write some code that can generate a Minecraft Settlement for an unseen Minecraft map. We accept submission in several frameworks, see below. You submit the code, together with some additional information to us before the deadline. We then apply this code to usually 3 competition maps selected by the organizers, which are not known before the deadline. The resulting settlements will then be sent to a range of expert and volunteer judges, who use Minecraft to look at and interact with the settlements, and then assign scores to each generator. We then announce the results - including who got the highest score and relating the settlements, code, and feedback publicly, both online and at a suitable scientific conference. Rules
Winners will be announced during the 2021 Foundation of Digital Games Conference. At the same time we will also publish the detailed evaluation results online for each algorithm. Once the results have been announced we will also publish the competition maps, the settlements generated by the algorithms, and the participants description of their algorithm. We also encourage participants to publish their code, but this will not be mandatory. Unless participants object to sharing their code, we will make it available on our website with an appropriate open source license. If they decline, we will not publish the code. In either case, the organizers will not take exclusive ownership of the code. You can find previous years maps, code, resulting settlement, evaluation scores and other things on our website. EvaluationThe submitted algorithms will be judged based on a set of 3, different, unknown maps. Example training maps will be provided. The competition maps will not be publicly available until after the competition. Every submitted algorithm will be run on the same 3 unknown maps, and the resulting maps with settlements will be anonymized and given to the judges. The judges will also be able to see the 3 maps in their natural state, before the algorithm was applied. We might change the number of maps if submissions are getting too high. The algorithms are given (dependent on the framework) a map and a 3d bounding box they should generate a settlement in. The algorithm should be able to run on maps of varying sizes - and should not run substantially longer than 10 minutes on a regular PC, and not exceed 2 Gbit of RAM for a single map. In the past we have usually used maps of around 256 by 256 blocks, that were generated by the standard Minecraft terrain generator. But the algorithm will be given the exact dimensions of the map when it is run and should be able to adapt to slightly bigger or smaller maps. For example, a competition map might be 200 by 300 in size. The competition is judged by a set of human judges who are either members of the advisory board or delegates selected by the organization team. If a member of the advisory board or the organization team ends up submitting an algorithm themselves, they will be excluded from the evaluation process for that year, i.e. they will not be allowed to nominate judges or judge themselves. The plan is to have all judges evaluate all algorithms and average their scores. If this is not possible, due to high number of submissions, there might be several evaluation rounds, where earlier rounds are used to reduce the number of submissions by evaluating a lower number of maps with only a subsection of the judges. The judges will then evaluate all 3 maps for a given algorithm by spawning at the central point (for the x and z coordinate) of the map. While the judges are in creative mode, they are encouraged to walk through the settlement from a survival perspective. They are given a set of evaluation criteria (see Evaluation Criteria) and are asked to assign points based on the performance of the algorithm across all maps. The sum of the points is the score of the algorithm. The algorithm with the highest score wins. New Evaluation Methods in 2021For 2021 we plan to add two modifications to the evaluation in comparison to the 2020 maps. First, we will include one larger map, possibly around 1000 by 1000 blocks. Here the challenge is to a.) make sure the algorithm still finished in the allotted 10 min. time window and b.) to figure out where to built on the map. Judges will spawn in the middle point of the map, and the generators should ensure in some form (sign, road, etc.) that judges are able to find the generated settlement on the map. Secondly, since we want to move more towards co-creativity we will have one map that has already some structures build on them, similar to a player starting a settlement. We acknowledge that this is challenging, and are interested in how this can be dealt with. Participants are allowed to ignore, destroy, modify, or built over the existing structure, but we assume that judges will reward adaptivity to existing structures in their scoring. Evaluation CriteriaWe defined a set of evaluation criteria based on what we believe the challenges in moving towards more human-like settlement generation are. So, when evaluation the generated settlements judges are asked to score each generator in four categories: Adaptability, Functionality, Narrative and Aesthetics. For further details, read on at 'Challenges' below. Chronicle ChallengeWe are also still running the optional bonus challenge: Chronicle Generation. This is an optional challenge, that is evaluated separately from the regular competition. Contestants will have to indicate that their entry is competing in this challenge during submission. SubmissionSee valid submission methods page to see all current methods. ChallengesThis section is about the specific challenges in generating a Minecraft settlement on a given map. We believe that these are some of the core obstacles that need to be overcome to move closer to human-level design. We should point out, though, that this list is not exhaustive. It is likely that there are other challenges and obstacles to get to human-like settlement generation. Furthermore, we consider the challenges outlined here ambitious goals to move towards, and do not expect them to be fully solved right away. Adaptation to Environment and TerrainReal-life human settlements are adapted towards their surrounding environment and terrain in multiple ways. On a per-building basis, houses and structures reflect the climate conditions they are built in. Furthermore, the environment of a settlement might also dictate the available resources for construction, and certain conditions might require specific types of buildings, such as irrigation, shelter, etc. A house in the cold climate of Edinburgh, for example, might be built sturdy, to protect against wind and cold and keep the warmth in. Likewise, the easy availability of stone might influence the choice of building material. Real-life settlements are also adapted to the immediate terrain they are in. Natural features, such as mountains or lakes, play a role in the selection of where to settle. Some terrain features can be both positive and negative for a specific settlement, and settlements are often built to take advantage of the benefits and to compensate for the problems. A river, for example, might provide more food and added mobility via ship travel, but might also limit mobility by foot, and be a potential flooding hazard. A settlement might subsequently be built close to the river (or even more likely, at a good location to cross the river), to take advantage of the benefits. The same settlement might also build a dam to protect against flooding, and houses close to the water might sit on stilts. If the settlement dumps waste into the river there might also be a development of different districts, where better, more desirable houses are further up on the river. On the other hand, humans change the terrain and environment in a limited fashion to better suit their needs. While it might be impossible to remove a mountain or an ocean, it might be possible to build a canal or tunnel. In summary, human settlements are shaped by and in turn shape the terrain and environment around them. If we study human-built settlements in Minecraft, we can see that these principles are realized in some existing builds. AI settlement generators, on the other hand, struggle to adapt to environment and terrain. They often rely on flattening large areas and then building settlements that do not reflect the surrounding map at all. Similarly, the houses and structures built by them are often templates that get placed as a whole unit, and are in no way reflective of the environment they are in. As part of this challenge, we encourage participants to develop generators that produce different settlements for different kinds of maps, reflecting the available materials, surrounding terrain and environmental conditions. In particular, successful entries should generate settlement that fit into the topography of the map without changing it too much, and build structures that fit into the topography. We also encourage participants of the challenge to think of other ways in which the given maps influences the generated settlement, ideally in such a fashion that it is evident to a human observer what kind of adaptation happened. Ideally, different maps should lead to different settlements, and it should be clear in which way the settlements where shaped by the map. Also note, that there is some overlap with the other three criteria, as their particular solutions are also often ideally adapted to the underlying map. Functionality from an Embodied PerspectiveMinecraft is not just a creativity tool, but also a game. Subsequently, a Minecraft settlement is not just an aesthetic artifact; it also needs to fulfill certain functional roles. One big aspect of this is accessibility and mobility. Is it actually possible to walk through the settlement and reach its different parts? Are there doors into the buildings, are the bridges over the river? How easy is it to navigate the settlement? A second big function a settlement provides is protection from various dangers. Is the settlement well lit, or does it otherwise prohibit mobs from spawning? Does the settlement manage to keep dangerous mobs outside? Are there other forms of defenses and traps to protect the player? Finally, there is also a need to obtain food and process it - so is there an accessible way to produce food and not starve? This list of functional requirements is not exhaustive - it is just meant to give some examples. This criterion does have a certain overlap with the challenge of adaptivity. Certain functionality only needs to be provided in some cases. For example, bridges compensate for reduced mobility introduced by obstacles. They are an adaptation to reduce the negative effects of the terrain. Other functionality takes advantage of given terrain features, such as river lock further enhancing the mobility of an existing waterway. This should also be considered, not just from a player perspective, but also under the assumption that the NPC villagers living in the settlement have similar needs. There is crossover here with the next section, because some functionality displayed in typical human-build settlement is not functional in the strict sense. For example, in reality a lot of houses have pitched roof to ensure better rain water runoff. In Minecraft, this is not necessary (as rain does not pool on the ground), but players still build houses with pitched roofs. This narrative functionality alludes to a functionality we understand, yet is not strictly functional in Minecraft. We still encourage the inclusion of such functional elements. Ideally, these narrative functionalities should also be adaptive to the environment. So, pitched roofs should be more popular in a rainy swamp than in a dry dessert, etc. In summary, the fact that there is an actual player in the world gives us an embodied perspective. The avatar can interact with the world in a variety of different ways, and a settlement can provide different affordances to the player. The challenge in this competition is to produce a generator that can ensure that the settlement provides a maximum of functionality and affordances for the player, while also satisfying the other criteria. Believable and Evocative NarrativeHuman settlements are not just collections of functional buildings, but they also tell a story about how they came about, who are the people living there, and how they see the world. When we look at human-created Minecraft settlements, we can often see how certain human or imagined cultures are reflected in the created buildings. There are cities that resemble ancient Rome, mythical Elven forest outposts, and modern US cities. Often, the buildings also reflect very clear narrative ideas, such as “this is a defensive mining outpost built in a harsh environment”, or “this is a capital city, built to impress foreign and domestic visitors alike”. These settlements are evocative, i.e. they manage to transport this story by looks alone. Relating to the earlier challenge of adaptivity, the better settlements also evoke narratives that work well with the terrain and biome they are in. The narrative they evoke should fit the terrain and environment of the given map, and ideally arise from it. Different maps should result in different narratives. Human settlements also reflect how they came about. Medieval cores of modern cities, for example, tell a part of a settlement's origin story. This is something rarely seen, even in human-made Minecraft builds, as settlements here are created with the final state in mind. But procedural content generation does have a tradition of simulationist approaches, which would be a possibility for this challenge. An AI could simulate people living and building a settlement over several stages, rebuilding or replacing structures, or slowly modernizing them. This could solve several of the previous problems, as buildings and the overall settlement would be the actual result of an adaptive process, and the produce of an actual sequence of events forming the basis for a possible narrative. There is also a certain overlap between narrative and functional requirements. How easy the player can obtain wood for building and further process it is a functional requirement. To satisfy this a settlement could have a nearby forest and a workbench nearby. But to make this work with a narrative of a settlement that relies on wood production it might also be good to build a structure that looks like a sawmill, and a street to transport the goods to town. This would not produce any additional functionality to the player (or the digital villagers), but it would tell a narrative that aligns with the functionality provided to the player. Similarly, there could also be elements that have purely narrative functionality, i.e. structures that have a believable fictional use that is not immediately reflected in the game mechanics. For example, a settlement could have an aqueduct to provide water, even though Minecraft characters do not need to drink. To summarize, the particular challenge here is not just about narrative generation, but also about generating a narrative that fits with the generated settlement, and ensuring that this narrative is communicated to the player with the generated structures. Inspiration by existing, historical or imagined cultures can help to transport this narrative. The aim is to produce something that is both believable and evocative, while still being aligned with the previous criteria. Visual AestheticsAesthetics are arguably subjective, yet architects and city planners usually follow a range of principles when it comes to designing a settlement or a house. While untrained humans manage to intuitively realize these principles with some success, it is difficult to design an algorithm with automated aesthetic judgment. Existing Minecraft mods that add structures to the world circumvent this problem by hand-designing appropriate templates, and then building a settlement out of those templates. This solution has several problems: it allows for little variation between the buildings, as they all need to be pre-designed, and it also allows for little adaptation of the buildings to the surrounding buildings, the underlying terrain, etc. Buildings are sometimes parameterized, and hence reflect some specific environmental settings, such as available materials or the climate they are in. This makes them more adaptive, but there seems to be trade-off between controlling the exact look of a structure, and making it adaptable towards uncontrolled environmental factors. The challenge in our case is to ensure that buildings still follow basic design principles while being adaptive, functional and evocative. There is also the further problem that the overall settlement itself should follow certain aesthetic rules. How buildings are aligned and what sight lines exist can play an important role for the overall feel of the settlement. Finally, there is also the question of how well the particular aesthetic expression chosen aligns with the other challenges. For example, a settlement which has a narrative of being a foreboding fortress, and is designed with that functionality, should also have an aesthetic that reflects this. Bonus Challenge: ChronicleWhile cities implicitly tell a story about their inhabitants, cities also often have explicit stories about their history. These chronicles contain notable events that affected and shaped the settlement, such as fires, wars of periods of vibrant trade. The can also provide a lot of additional context, such as what was the function or reason for the settlement during different time periods, or what ideas and philosophies have shaped the settlements development. They also often contain stories about important people that shaped the city. Basically, anything that could be written about a city and would be achieved in the city is appropriate. For this challenge, your task is to create a chronicle that is currently stored in the city. It should be written as a Minecraft book, and should contain text relating to the generated city. This challenge relates back to several of our earlier criteria, such as adaptivity and narrative. The chronicle should be adapted to the actual generated city, and should the story of that specific city. The less generic this story is, the better. The chronicle challenge is optional, and entries will be evaluated based on the overall quality of the chronicle, and how well it fits with the different cities generated. ExamplesHere is a link of videos of good, human-created settlements that showcase some of the things we are looking for in a generator. FAQHere is a short FAQ. Entry DescriptionsSubmission descriptions are taken from the user-input "about" box or written by the authors themselves. Porter HickeySubmitted by Alex Porter & Matthew Hickey, Computer Engineering Class of 2021, Memorial University of Newfoundland. Brianna & KyleWe aimed for a natural-looking settlement that adapts to the given environment as much as possible while also being functional for players and aesthetically pleasing. William, Selina, Ho Kiu, RhysDemo Video Cristopher YatesThis generator is an MCEdit filter (and several schematic files). Although there are three options that can be changed in the MCEdit GUI, I strongly recommend keeping them at their default values. KeywarnMy generator uses an agent based method. Agents explore the area by themselves, labelling valuable building sites and the materials found in those areas. Builder agents then create buildings and structures, laying down roads as they go. aith (solo-team)My entry is an iterative agent simulation that uses human-like behaviour to create organic settlements. It also generates a chronicle for the chronicle challenge! It uses Nils' HTTP Server. For a detailed explanation, see the WIP paper draft! I'll include an example save, where a large settlement is generated at position (88, 68, -52).
Nils GawlikWith this generator my goal was to create something that makes use of image processing algorithms and explores to which extent they can be used for this competition. There are different steps to the generation process, but ultimately they are all variations of the same concept: Interpret the build area as a 2d image and then use image processing algorithms to transform it into something new. Basically all operations in this generator are done in this pixel representation, including operations which you might normally think of as generating discrete objects - like placing doors or building staircases. LIGCPython The World FoundryThis generator creates a dungeon settlement under the surface of the Overworld. The settlement is also a playable game environment, with the player tasked with recovering treasure and relics from the vaults in the deepest part of the dungeon. LisaThe main concept for this generator is to simulate the developing village as an entity with needs that can be fulfilled by the different buildings. The different needs that this village has are: food, housing, jobs, resources and social. The different buildings that can be built fall into two categories: the buildings that use the “house” template with different utility items inside, and the buildings that do not use the “house” template. Buildings that use the house template are: house (satisfies housing), cleric, crafter, smith, blacksmith and enchanter. All aforementioned buildings but the house satisfy jobs and resources in different ratios. The buildings with different templates are farm (satisfies food), fountain (satisfies social), and pasture (satisfies food and resources). A special case is the tower: it fulfills no function, but it is the first building that is placed during the simulation, and therefore marks the spot the village is growing from. It also contains the chronicle. MAU_AE_JF[NOTICE: this generator is not optimized to handle the 1000x1000 area (it takes several hours) it can complete the 256x256 area in approx 9 minutes on a home computer with a 3.6 GHz processor.Also if the settlement is far from the center of the selection of selection a white arrow above the center points towards the settlement] Rainbow ParliamentHello! We are a group of high school students from China! We failed to include our chronicle in the chest inside of map, but we still would like to share this fascinating story with you guys: In 1991, the world was shocked by the August 19 coup in the already unstable Union of Soviet Socialist Republics (USSR). The hard-line faction of the Central Committee of the Communist Party of the Soviet Union (CPSU), formed to prevent the "liberalization and federalization" of the Soviet Union by Gorbachev, General Secretary of the CPSU, and Yeltsin, Secretary of the Moscow City Council, marched the army into the center of Moscow on August 19, 1991 and declared martial law and a curfew, while the state security forces involved in the coup arrested and detained many officials deemed "threatening. "On August 20, due to liberal opposition and a prolonged standoff, the hard-line Soviet defense minister, Marshal Yazov, ordered "Operation Thunderbolt". The Emergency Committee immediately called on the independent and non-independent republics and armies to unite with the Emergency Committee in order to preserve the unity of the Soviet Union. The following day, serious bloody clashes between the army and citizens in the streets were brutally suppressed by the overwhelming superiority of the army, and order in the streets was restored on August 22. Gorbachev, who was scheduled to return to Moscow on the 20th, arrived at Sheremetyevo airport from Crimea in the early hours of the following day after hearing about the coup on the 19th, only to be arrested by NSDAP troops who had been waiting for him. Threatened by hardliners, Gorbachev declared a state of emergency in the country. With the General Secretary and the central government in the hands of the hardliners, the liberal wing of the Communist Party of the Soviet Union lost its power, and some officials fled to the West, while others who did not make it were imprisoned by the KGB in Lubyanka and the Gulag on charges of treason. The Central Asian steppes were home to Muslim communities. The Soviet Red Army's "stabilization" operations in Central Asia were much easier than in the nationalist Ukraine and the Baltic States, which were backed by NATO in the west. The Central Asian states and pro-Soviet Belarus, still somewhat nostalgic for the Soviet Union, were quickly stabilized, but an organized armed rebellion broke out on the Caucasus and European borders until November, and scattered guerrilla resistance in the forests and mountains continued into the following year. On September 1, 1992, after a series of internal conflicts and political purges, the battered and almost disconnected Soviet Union announced to the world for the first time that the rebels of the bourgeoisie, right opportunism and foreign intervention had been defeated. The Soviet Union retained all its territories except Estonia, Latvia and Lithuania, and Yanayev, the former Soviet vice president and hard-line political leader, became general secretary of the Central Committee of the Communist Party of the Soviet Union, while Gorbachev was imprisoned and exiled to the United States. The fragility and instability of the "new" regime led the Communist Party to adopt a tightly planned economy, the existence of the ruble was temporarily abolished, the concept of food stamps was reintroduced, and the country returned to the wartime communism of the Civil War. the huge Soviet economy a severe blow Rexos and XeonoXAsk me on Discord, we have a lot of things to say and not enough time to write before the deadline: XeonoX#8802 Tsukuba TeamThis Minecraft Settlement Generator is generating a coherent minecraft village. In that village, there are unique interactions between villagers using books in their house. There is a system of death, with a death record in the village and a cemetery. In the Town Hall, there is a special book presenting the village, so feel free to get into the Town Hall first. Murders often takes place in the village, so be sure to look around for any clue. Beautiful_KaaskodersOur village is inspired by old dutch cities like: Naarden, Den Briel (aka Brielle), Hellevoetsluis, Leiden and the old city centre of Amsterdam. (and many more…) It is generated as follows: As an easter egg to Leiden, we have added exactly one Iron, Redstone, and Lapis Lazuli block to the streets of our city (although some of these might be buried beneath the outer wall). All our filters have the prefix 'Beautiful_' to allow us to more easily find our own filters and test them, this also became a running joke during development, hence our team and generator names Leiden University MGAI team 1This code creates a procedurally generated city based on Eixample, Barcelona. CharlieMDRzI expanded on my team's submission for GDMC 2020 named LesCharretiers. In summary, the steps of my settlement algorithm are the following: HunternifMy project is a medieval castle generator. The current implementation uses something like gradient descent to detect mountains. Then it selects the biggest mountain and puts and arranges towers around the mountain's perimeter. The towers are connected together by walls. Finally, each tower has a smaller turrets built on top of it, placed at random angles and sizes. The last part was inspired and partly copied from Zivbot's Grand Castle Generator Terje SchjelderupI wanted to go for a top-down approach, like a city planner or architect would do for planning a new town or city. Going from the input world, to an area plan (where to put the town) to placing roads, filling out the area with more roads, splitting the area between roads into plots, and finally building houses (and maybe other structures) on those plots. The hierarchical structure of the generator resembles a layered network stack, in that each task of the pipeline acts only at its own abstraction layer, and changes at one layer mostly affects the layer below and above only. Hopefully this has resulted in a generator that is easily extended in the future, which I hope to do for future iterations of the GDMC competition. ICE_JIT_ft_WFCSome of you might want to but still cannot make a trip to Japan these days. Here we proudly present a solution to you through this competition platform. Into our settlements, we have tried our best to introduce all the very, very Japanese aspects and elements, including Japanese-style rowhouses, bridges, two-storied shrines, shrine gates, and five-storied pagodas + even hidden (transparent) Ninja treasure chests. Our settlements should also be mysteriously beautiful at night due to light up, which helps prevent mobs from spawning. This generator is a slightly enhanced version of our 2020 generator ICE_JIT, by introducing buildings generated by Wave Function Collapse (WFC). Results
|