Autodesk’s AI Innovations Transforming Sustainable Design and Construction

TileGPT is an experimental AI prototype developed by Autodesk researchers to augment a designer’s creativity. This results in novel site plan designs that balance a team’s goals for carbon, livability, and profit, all generated through a no-code, natural language user input.

• Innovating materials data and impacting project timelines: Autodesk’s AI-assisted design technology is making significant strides in the Architecture, Engineering, Construction, and Operations (AECO) industry, harnessing data to automate tedious work and speed up processes. With its tools, the company is addressing unique design and net-zero challenges faced by their customers.
• Industry leader in AI and CAD geometry publishing: Autodesk has a significant history in generative design, having published its first paper on the topic more than 15 years ago and papers on generative AI since 2017.
• Enabling low-carbon, early-stage design decisions: Autodesk Research is developing prototype AI tools like TileGPT that use material data to help augment users’ design skills and realize sustainable solutions.

For centuries, scientists have studied crystals: As early as the 1600s, they researched them to understand materials science and solid-state physics; today they use them to build countless technologies, from silicon chips to batteries to solar panels. And throughout the centuries, it was widely believed that the number of unique crystalline formations in existence was fixed at approximately 20,000. But this year something changed: 2.2 million crystalline structures were discovered.

With this breakthrough comes a world of new possibilities in sustainable electrification technologies. Electric vehicles can now go farther between charges, photovoltaic panels can provide more power more efficiently, and countless new technologies we haven’t yet considered may soon be possible.

It’s all because of GnoME (Graph Networks for Materials Exploration), a deep-learning artificial intelligence (AI) tool that uncovered the equivalent of eight centuries worth of human knowledge in roughly three years. In the words of researchers at The Materials Project and Google DeepMind, this revelation—discovered through automation—had simply “escaped previous human chemical intuition.”

Admittedly, GNoME’s discovery is AI at its most ideal: navigating vast amounts of data faster than any group of humans could conceive. The deep-learning tool identified and built on existing patterns, establishing exhaustive new possibilities by training on available datasets. GNoME is a clear example of AI’s ability to perform—and excel at —tasks typically requiring human intelligence.

Despite this example, artificial intelligence isn’t always so clever. While ChatGPT has sent the concept of AI on a meteoric ascent up Gartner’s Hype Cycle, its cultural ubiquity has led to all-too-broad applications with the expectation of easy solutions that just don’t exist.

However, like GNoME’s recent discovery in crystallography, some AI applications do have the capacity to make significant strides in specified areas of study. Architecture, Engineering, Construction, and Operations (AECO) is one of them—and Autodesk is developing the very tools that will give artificial intelligence the ability to solve the unique design and net-zero challenges our customers face every day.

Step back from the headlines and the hype

Autodesk has a significant history in generative design, having published its first paper on the topic more than 15 years ago and papers on generative AI since 2017. The Autodesk Research AI Lab has released more than 65 peer-reviewed academic research papers about AI and CAD geometry, making it the world’s leading publisher on the subject. In addition to issuing research papers, Autodesk has sought to advance the entire field of AI research by open-sourcing machine-learning datasets in collaboration with MIT.

Photographers, TikTokers, and Hollywood scriptwriters aren’t the only ones grappling with the impacts of AI technology–our customers are too. Autodesk’s 2024 State of Design & Make report was definitive: 75% of the companies surveyed will increase spending on AI over the next 2-3 years, with 66% of industry leaders agreeing that AI will be essential for business. Respondents also revealed that sustainability is a top priority.

Autodesk is continuing to significantly invest in the future of AI, both through funding computing resources and hiring talented AI researchers. Consequently, Autodesk is leading the way in exploring the real-world potential of AI technology in the Architecture, Engineering, Construction, and Operations (AECO) industry—and helping customers achieve their sustainability goals.

This Carbon Story blog series outlines the urgent necessity of moving the needle on sustainable solutions in AECO with tools that our customers can use today. In Part 1, we explored novel materials currently in development for scalable, net-zero applications. In Part 2, Autodesk Research shared how the results of those experiments are driving generalizable material software approaches through accessible data.

Now, in Part 3, we will demonstrate how the data from those experiments is being tested in real-world AI prototypes to enhance a designer’s creative process. These AI tools will result in significant changes and opportunities to our customer’s workflow by:

1) automating repetitive work,

2) analyzing overwhelming amounts of complex data, and

3) augmenting their design teams’ creativity and problem-solving capacity for actionable, net-zero insights.

Section 1. Automation: AI as a trusted tool

To understand automation, we must first confront one of AI’s most critical challenges: producing accurate, validated results.

Optimistically, State of Design and Make report respondents were overwhelmingly credulous toward AI, with 76% surveyed indicating they trusted the technology for their respective sectors. However, more in-depth interviews with industry leaders revealed an undercurrent of skepticism and an understanding that warrants a prudent approach.

The industry is cautiously expectant, making trust and accuracy of AI tools more important. Unfortunately, this goal can prove a significant challenge when working with large language models (LLMs). They carry the potential to perceive patterns that result in nonsensical and false outputs, known as hallucinations. For example:

Prompt: “What was the first building in the U.S. to use mycelium composite as a permanent facade material?”

ChatGPT: “The first building in the U.S. to use mycelium composite as a permanent façade material is 75 Nassau Street in New York City.”

While ChatGPT offers a rapid response, it’s flatly incorrect—on a few levels.

As readers of this blog series know, Autodesk Research is working with industry partners to build the first permanent façade in the U.S. made from a mycelium composite. But ChatGPT gets it even more wrong: 75 Nassau Street doesn’t exist. It is an empty site, slated for development in 2027. The average user might hypothesize what caused this hallucination: Is it because the planned tower on this site is vaguely fungal-looking in easily searchable renderings? Or could it be that there was a three-star review for a “mushroom omelet” served at a now shuttered diner near the site back in 2012?

Scientifically it is understood why LLMs are generating false responses by replicating the syntax of language. “Hallucinations in LLMs result from the fact that they are trained to predict the next word—that’s it,” explains Tonya Custis, Senior Director, AI Research at Autodesk. “LLMs effectively learn the syntax of language, not semantics or facts. We can understand why a language model could see an address, know what it looks like, then make up a fact about it.”

Ultimately, the reasoning that brought ChatGPT to this specific false response is a black box. While similar occlusions may be acceptable in more mundane or even frivolous AI prompts—say, to visualize Pope Francis decked out in Balenciaga drip—such errors would have far more serious consequences if a structural engineer were to rely on this output to design a building or bridge that can withstand hurricane-force lateral wind loads. “Close” isn’t good enough.

“One key challenge lies in the probabilistic nature of deep-learning AI models,” says Mike Haley, Autodesk Senior Vice President, Research. “Nobody wants to drive across a bridge that was [designed by AI] to be approximately correct.”

But there’s good news, Haley continues. “Considerable effort is underway to realize methods for controlling, validating, and adapting generative AI models to produce precise and controllable outputs under specific circumstances,” he says. One way AI can successfully do that, Haley explains, is by learning the well-defined patterns of designer’s repetitious work in order to reproduce it. “This will result in automating laborious tasks like producing drawing sets, finding the right engine components, or checking the building code compliance of an architectural design.”

Imagine the innumerable complex systems a design team must contemplate when designing a building. There are quantitative aspects, such as the function of a space and efficiency of its mechanical layouts, as well as qualitative metrics, which can vary from aesthetics and livability to how a structure relates to its natural ecosystem. Now consider the sheer number of additional restrictions that govern a project’s design choices: sun exposure, zoning ordinances, building codes, fire safety, energy use—the list goes on.

Every single one of these decisions must be considered, validated, and revised in a repetitious, often tedious process to comply with a staggering number of constraints. By encoding data on how these repetitive tasks are typically completed, Autodesk design and make tools could automate that tedious bucket of work, freeing designers and engineers to shift their brainpower toward a more creative focus.

The Autodesk Research AI Lab is focusing on precisely this, says Custis. “Our research intends to understand as much as possible about design intent and how to manipulate the geometry in order to achieve that,” she says. “For people to have freedom from the tedious details of generating the geometry, the models we are training must embody and understand the geometry incredibly well. Getting the geometry correct and precise while making it intuitive for the user is our number one priority.”

Section 2. Analysis: AI for actionable trade-offs

Autodesk Research is developing AI-assisted tools that can provide rapid analysis of complex, competing factors. Why? Speed is crucial in design and construction, especially in emerging, fast-growing markets. AEC firms often need to submit concept designs just to qualify for the opportunity to compete for a project. Maximizing productivity while minimizing labor hours creates a competitive edge in these markets.

The production of housing, from financial entitlements to completion, also tends to favor the swift, especially when government grants are on the line. For instance, the California Housing Accelerator supports the construction of new affordable housing by providing massive incentives to projects that can achieve ambitiously short timelines. In the first year alone, the program has already “accelerated” nearly 5,000 housing units.

One example is The Phoenix, an affordable housing complex in West Oakland that will feature 316 of these accelerated residential units. Designed by a multi-disciplinary team that included Autodesk Research, The Phoenix utilized Autodesk AI-assisted design tools and the power of the Autodesk Platform to build more sustainably—and faster.

“Meeting the time challenge can unlock billions of dollars in subsidies to build new housing, but it requires creative solutions at every step,” says David Benjamin, Director of AECO Industry Futures, Autodesk Research, who led Autodesk customer engagement in development of The Phoenix. “The innovation required to qualify for such funding starts with design.”

Beyond speed, housing poses a multifaceted challenge with a diverse and often competing set of stakeholder interests. Far too often, decisions are made from the perspective of short-term cost savings at the expense of longer-term gains stemming from increased occupant enjoyment and livability. The complexity quickly arises by making many decisions that seem to work independently but impact each other to create third- and fourth-level ripple effects of unintended consequences.

The process of designing The Phoenix was an exercise in balancing all of those trade-offs and their potential ripple-effect scenarios—not via hours of exhaustive human labor but seemingly instantly, by AI-assisted, analysis design tools, which weighed the complexities of factors such as total carbon emissions, project cost, and livability. Here, by using early-stage design tools such as Autodesk Forma, The Phoenix was built as the “best-case scenario” that satisfied all of housing’s multi-faceted goals—and stakeholder interests.

“Making the right decisions when developing a project with so many stakeholders and moving pieces gets overwhelming really, really fast,” explains Andrew Meagher, Vice President of Design and Engineering at Factory OS, the modular housing company developing The Phoenix. “Working with Autodesk Research gave us new insights into how all these pieces are connected. Instead of just relying on past design approaches or intuition, we could instead analyze the data and uncover these novel connected sustainable and livable approaches that meaningfully address California’s housing crisis.

Meagher continues with an example of one such data-informed decision at The Phoenix. “We wanted to get more sunlight into the courtyard space between buildings,” he says. Meagher’s team already knew that decreasing the building’s height along the western facade would create more sunlight—but the unintended consequence would be more highway noise for the majority of inhabitants. Conversely, spreading the buildings out to allow more light to reach the courtyard would result in more space for residents—but could cause costly delays in municipal approvals.

By running the AI-assistive tools within Autodesk Forma, Meagher’s team discovered the best solution: Shift the Phoenix’s building massing and reduce its height in a few key—yet unexpected—areas. What’s more, the site adjustments in this solution resulted in another benefit: more privacy for residents.

“We found that making these geometry moves would greatly increase occupant livability with minimal impact to construction cost—and studies have shown that if residents are happy, they are more likely to renew their lease, which results in increased net operating income,” Meagher says. “On top of that, this design move also lowers carbon emissions by reducing the energy and utility cost used by the residents.”

Here is where AI is an invaluable tool. Functioning as an expert analyst within the design team, it can produce novel solutions to uncover actionable insights and generate higher-performing site plan layouts than could be created by a human designer alone. It has the potential to predict the ripple effects that they may cause, not only in terms of construction cost and energy use, but also more intangible factors such as future revenue potential and occupant livability.

The resulting AI-assisted Project Phoenix layout far exceeded the customer’s goals toward minimizing total carbon, maximizing operating profit, and increasing inhabitant livability beyond the experienced design team and the developer’s initial assumptions. “The Phoenix is a real-world demonstration of how new, AI-assisted technologies can transform traditional AECO workflows in dramatic and impactful ways,” Benjamin says.

Limitations in analysis

Now, imagine if we add even more factors. Make the facade waterproof. Make it acoustically insulative. Make the surface geometry to maximize reflected light. Make it installable in six days—not six months. AI can analyze and make recommendations for all of these commands.

But what about: Make it out of a novel net-zero material like mycelium? Alas, this is one command that AI can’t quite yet answer because the data for this scenario does not yet exist.

“We need to pivot towards AI systems that do more than just replicate past designs,” says Adam Gaier, Principal AI Research Scientist, Autodesk Research. “We need to develop AI tools that are capable of generating novel solutions that prioritize sustainability.”

This is how AI will soon augment our customer’s net-zero work.

Section 3. Augmentation: AI to expand the possible

Three Autodesk researchers are meeting over Zoom. On a shared screen, pixelated columns are quickly cascading down the screen, filling in a new and magical cartoon world. First, a slightly purplish-blue sky appears, followed by implausibly puffy white clouds. Then, the canvas is sprinkled with floating red-brick boxes. Rounding out the picture, a super-saturated green pipe emerges, ejecting an angry-looking bipedal mushroom. It can no longer be denied: This is a level from Super Mario Bros.

But this is a level unlike any that has existed before—just one of infinite potential layouts generated on the fly by Gaier, along with James Stoddart and Lorenzo Villaggi, both Principal Research Scientists, Autodesk Research, based on their written commands. Directives like, “No pipes, some enemies, many blocks, and low elevation,” produce an easy level. But how about “many pipes, many enemies, high elevation?” Now, that’s a challenge.

This experiment is based on a paper, released by a team of IT University of Copenhagen researchers, that demonstrates a method for using GPT-2 to both encode and generate new and novel game levels for Super Mario Bros. Appropriately, they named their AI creation “MarioGPT.” It is also the jumping-off point for a new experimental prototype Autodesk AI technology that explores how designers can make low-carbon, early design stage decisions.

“MarioGPT showed us that GPT models could be trained to produce coherent 2D architectural layouts in the same way they generate coherent text from a user prompt,” explains Stoddart. “But could we take this further? What if those blocks, pipes, and Goombas were instead design elements we could then use to compose buildings and landscapes?”

Furthermore, could Autodesk Research adapt the underlying AI technology so that instead of generating a superficial novelty world, AI instead leads to more sustainable, impactful design workflows?

Stoddart continues: “To a GPT model, all that matters is learning the high-level relationships between tokens, whether they represent words in a sentence or parts of a design layout. We wanted to go even further. Could we also teach the model which designs were high performing? Could we then train a GPT so that any user—regardless of their level of experience—could then use a prototype Autodesk AI to design a site with carbon-sequestering parks and high-density, low-carbon housing units that can balance great views with privacy?”

The answer is yes. Dubbed “TileGPT,” a new experimental proof of concept AI tool developed by Autodesk Research does just that. Users can enter their project goals as simple prompts such as “a site with many parks, minimal total carbon, and maximized net-operating income” or “a site with low embodied carbon, high resident livability, and few parks.”

Based on these directives, TileGPT augments users’ design skills to realize sustainable solutions by quickly filling the screen with a manifestation of this site plan. The user can then highlight sections of the site—perhaps the buildings are too close in one particular area (reducing unit privacy and negatively impacting overall resident satisfaction)—and request new iterations for only that specific area. Or they can scrap the whole thing and quickly try out another high-performing version.

The user and AI develop a dialogue with each other, seeking solutions together. There is an almost magical feel to the process, one that hints at future development possibilities. The AI performs less like a separate passive tool, and more like a constant, tireless design collaborator—one you can have a literal conversation with.

“So often we hear members of the architecture profession worry that AI might be on a trajectory to replace them as design professionals,” says Patricia Ramallo, AIA, Assistant Vice President of Innovation with the National Council of Architectural Registration Boards (NCARB). “When Autodesk Research presented this TileGPT work to NCARB’s leadership and volunteers at the 2023 Futures Symposium in Washington, D.C., it provided a compelling vision for how AI could help produce sustainable and equitable designs while also enhancing creativity and innovation—the qualities architects are already really good at.”

Technical AI innovation

The process of TileGPT begins with a design prompt and ends with a model that can be simulated in Autodesk Forma. To see more about TileGPT, visit the project’s github page at: https://tilegpt.github.io/

TileGPT combines the best aspects of two longtime core AI technologies at Autodesk:

• Generative Design (or outcome-based design): goals and constraints are used to explore and discover many possible, high-performing solutions
• Generative AI: uses machine learning to create new content, such as designs, images, or text, based on patterns learned from data

The process of initiating a generative design system for a non-technical user can be intimidating. The thousands of resulting design options can be a challenge to parse, causing users to act more like advanced data analysts than designers, but the resulting high-performing solutions are accurate. Generative AI, on the other hand, is intuitive and easy to use, but the outputs are rough and approximate. TileGPT combines the best of both worlds.

“With TileGPT, we’re exploring how AI design partners can learn winning design strategies by training them on the vast quantity of data produced by generative design,” Stoddart says. “This keeps the designer engaged in the design process with AI assistance enabling faster exploration and iteration to achieve their project goals without compromising performance.”

Adam Gaier presenting research prototype TileGPT as an early-stage carbon tool during the keynote at the AI in AEC conference in Helsinki, alongside other AI-focused work from Arup, Trimble, the University of Zagreb, and the University of Singapore.

The necessity for material data

AI needs data. ChatGPT was trained on 300 billion English words pulled from the Internet. GNoME was fed 600 years’ worth of scientific data in crystallography. Autodesk’s TileGPT research built a new synthetic training dataset using tens of thousands of outputs from generative design. In each of these instances, there is an implicit danger of reinforcing existing blind spots and merely replicating the same results over and over. The danger of looking backward and replicating the same old methodologies is even greater when we consider the many dire environmental challenges that demand forward-thinking solutions.

“Human designers often lack intuition on how to use new materials or create sustainable layouts,” says Gaier. “Generative AI built on existing data will reinforce biases toward traditional, non-sustainable, carbon-emitting materials. We need to develop deep knowledge of how these materials behave so that we can be confident enough about our AI suggestions to even offer them.”

TileGPT leads us in the right direction, but we must go further. To do so, we need to generate new low-carbon-material data for the AI augmentation systems of the future.

An optimistic future

That brings us back to where we began this series: researchers methodically testing, failing, and then testing again in dogged pursuit of viable net-zero material approaches to design and make. Together with our industry partners, Autodesk is scaling these solutions up to make an impact today. Researchers Brian Lee and Allin Groom, along with an impressive number of talented Autodesk colleagues, are all working together toward a shared net-zero future vision.

Material innovation doesn’t stop at discovery; real-world application is the real test. AI is poised to change everything about how we design and make both materials and buildings. One day soon, you may ask a GPT, “Why is mycelium composite used more often than Styrofoam in California housing projects?” And the AI response will not be a hallucination, but rather a statement: “Because Autodesk and their customers made this possible.”