The Solution to AI’s Alignment Problem is Precise Guardrails on How We Use It.

AI has a major alignment problem: the current AI architecture cannot ensure that AI systems will act in a way that is congruent with the interests and intentions of humanity. Case in point: this month, Anthropic reported that some of its agents took some illegal actions that exploited flaws on various government websites. There is no obvious fix to such problems. The solution to align AI lies not in requiring hard guarantees from the technology itself, which is practically impossible, but instead by establishing expectations and putting guardrails around the use and consequences of AI-based systems. A combination of government legislation and company self-monitoring to prevent foreseeable harms is urgently required as the technology continues to create new and unforeseen uses at a blistering pace.

Our position is based on several years of experience building a publicly available complex multi-agent system called DBOT that is designed to mimic the thinking of a stock valuation guru, our NYU Stern colleague Aswath Damodaran. DBOT’s agents do simple tasks. One agent translates a company’s expected growth rates, operating margins, reinvestments, and cost of capital into a price per share. Other agents perform tasks such as comparing a company to its industry peers, doing sensitivity analyses, report writing, and so on. In concert, the agents interact to produce a valuation. For example, DBOT valued the SpaceX IPO according to multiple scenarios ranging from highly optimistic to pessimistic. It struggled to justify anything above $100/share compared to the offering price of $135, well below where the stock trades today.

Here’s the rub. If you ask Damodaran himself to value SpaceX multiple times during the course of a day or two when there is no new information, he will give you the same answer. However, if you run DBOT multiple times on the same stock, it can produce different valuations and narratives. While we invested considerable effort to minimize these two variances, it is impossible to eliminate them entirely. The problem stems from the dominant Transformer architecture that underlies LLMs, which are designed for conversation in which some amount of variance is often considered desirable by developers.

Read the full Tech Policy Press article. 
___
Vasant Dhar is Robert A Miller Professor of Business and Professor of Data Science at NYU Stern. João Sedoc is Associate Professor of Technology, Operations, and Statistics at NYU Stern.