TECH

Developer shows off Red Dead Redemption 2, GTA V, Fallout 4, and other AAA PC games running natively on iPhone 18 Pro

A developer has built a Windows PC game emulator for iOS that seems to run AAA titles like Red Dead Redemption 2, GTA V, Fallout 4, and more at 720p 30 fps or better. The emulator is a port of Wine along with ARM64 and DirectX-to-Metal translation layers. The requires JIT to function well, so a manual sideload is needed and is preferably used with DRM-free titles.

Modern Apple iPhones have incredibly powerful SoCs but very few apps and games tap their full potential. One area that can let chips like the A19 Pro and A20 Pro unleash their full potential is game emulation.

Quite a few game emulators like the Delta for Nintendo systems, PPSSPP for the PlayStation Portable, and Dolphin for the GameCube and Wii among others are either available via the App Store or can be sideloaded.

However, there have been no emulators for Windows PC games on iOS unlike Android. That changes with Madeira.

Madeira allows running Windows PC games on a non-jailbroken iPhone. According to the developer Will Faust, Madeira is essentially a port of Wine (ARM64EC) for iOS with FEX-Emu for x86-64 to ARM64 translation and DXMT for D3D11 to Metal translation combined into a single pipeline.

Madeira requires a minimum of iPhone 13 Pro or above powered by the A15 Bionic or above.

In the project's GitHub repo, Faust claims that Madeira can run games like Thumper and Ultrakill while Marvel Cosmic Invasion "has reached gameplay" albeit being unreliable.

Now, Madeira also seems to be able to run the highly demanding Red Dead Redemption 2 (RDR 2) fully on device on an iPhone 18 Pro. While the initial attempt has been just a slideshow with single digit frame rates and loads of graphical glitches, the developer showed progress in attaining close to 30 fps in more recent builds at nearly 720p. There's still a long way to go, however, especially with regards to arrangement of onscreen controls.

Additionally, Madeira now supports a swap-like implementation that can offload memory into the iPhone's storage. This apparently kept the RDR 2's memory footprint around the 7.6 GB mark with possibility of lowering it even further to allow running on 8 GB iPhones.

GTA 5 support is in the works too alongside a host of other AAA titles...Faust also has a working copy of Grand Theft Auto V running on the iPhone. Performance seems to be decent for an initial implementation and will only get better with further optimizations.

The devs note in the project's Discord channel that using Steam hogs a lot of resources. The workaround for this was Madeira Dock, a headless Steam client that only uses the necessary verified Steam DLLs without the headroom needed by the full Steam client. This is not a bypass for Steam's DRM, however.

The devs note in the project's Discord channel that using Steam hogs a lot of resources. The workaround for this was Madeira Dock, a headless Steam client that only uses the necessary verified Steam DLLs without the headroom needed by the full Steam client. This is not a bypass for Steam's DRM, however.

The other alternative, they note, is to use DRM-free games from GOG.

The project has also demonstrated Fallout 4 running on iOS along with several other 32-bit games like Dead Space, Mirror's Edge, Far Cry 3, and more.

Emulation requires sideloading with JIT...Just-in-time (JIT) compilation is required for fast, real-time performance when emulating apps on a non-native host CPU. A major quirk with the Apple App Store (from iOS 18.4+) is that it does not allow apps and emulators to use JIT owing to security concerns, with the only exceptions being approved alternate web browser engines via BrowserEnginekit.

Therefore, the only way to get emulators like Madeira running is to sideload the IPA manually and enable JIT for optimal performance. Which means that apps like Madeira will never be distributed via the App Store.

Sideloading on iOS can be a bit tricky if you are new to the process, but it is not too difficult once you get the hang of it. 

You can purchase an Apple Developer certificate for $99 and get a year's time to use the sideloaded app before renewal. Regular Apple account users can also sideload provided you renew the sideloaded app every seven days.

mundophone

TECH

Google's September Pixel update proving to be a headache for some users

Google’s September 2026 Pixel update has reportedly caused lock-screen failures for a small number of Pixel 10 Pro XL, Pixel 8a and Pixel 8 Pro users, leaving some unable to access their devices.

Google’s September 2026 Pixel update was intended to deliver new features, security improvements and a long list of bug fixes, but reports indicate that it has also introduced a serious lock-screen problem for a small number of users. Pixel 10 Pro XL owners have complained that their pattern unlock resets after only two dots, while Pixel 8a and Pixel 8 Pro owners have reported problems with PIN authentication. The issue does not appear to be widespread, but it can leave affected users unable to access their devices.

Attempts to resolve the problem have produced mixed results. One Pixel 10 Pro XL owner reportedly tried rebooting, Safe Mode, connecting a mouse and reinstalling the same software build without success. A factory reset eventually restored access, although this erased locally stored data. Another Pixel 8a owner reportedly regained access after waiting for the "Too many attempts" lockout timer to expire. Google is reportedly aware of the complaints via its PixelCommunity Reddit account, although it is yet to officially comment on the issue.

The issue is particularly troubling because the lock-screen mechanism is also part of the security apparatus that protects user data, meaning that even Google cannot simply bypass it to recover your photos. This is why Google recommends erasing the device for users who are locked out, with backed-up data recoverable through their Google Account.

Google’s September update otherwise addresses numerous problems across Pixel devices, including crashes, connectivity issues and lock-screen notification bugs. Users who have not yet installed the update should either ensure important data is backed up before proceeding, or hold off on the update until Google has formally addressed the issue.

In plain words, your lock screen doubles as part of the key that unlocks your files, so nobody (Google included) can simply skip it to pull your photos out. That's why Google's own support page points locked-out owners toward erasing the phone, with the only silver lining being that anything backed up to your Google Account can be restored afterward.

As for Galaxy and iPhone owners, this one is purely a Pixel firmware problem, so there's nothing to worry about on your end. Our Pixel 10 Pro XL review scored its software an 8 out of 10, just under the 8.2 average for its price class, and a streak of bugs like this one won't help Google close that gap with Samsung and Apple.

Should you hold off on the September Pixel update? For now, this looks like a rare bug that isn't hitting every Pixel, so I wouldn't panic if your phone already updated and unlocks normally. But if the September update is still waiting for a restart on your phone, make sure your photos and files are backed up to your Google Account before you tap install.

Here's what to do if your Pixel won't unlock...Stop guessing and let the "Too many attempts" timer run out, since that's what got the Pixel 8a owner back in.

As a last resort, turn the phone off, hold Power and Volume Down until Fastboot Mode appears, then choose Recovery Mode.

From there, hold Power and tap Volume Up once, then pick "Wipe data/factory reset" and confirm with the Power button.

The one Pixel feature an update should never break...I understand bugs slip through, especially when the same build rolls out to everything from the Pixel 6 series to the Pixel 10 family. However, the lock screen is the one part of the phone we can't work around, and when the only way past it is wiping everything, the stakes are much higher than a laggy animation.

I'm hoping Google pushes a fix before more owners run into this, and I'd personally love to see it add a safer way to recover from a broken lock screen without losing a year's worth of photos.

mundophone

TECH

A $20 multimeter might reveal your CPU's overclocking potential

Can you predict how well your CPU will overclock by probing it with a multimeter? Before you start pulling out the Fluke, let's go over the details of this story, because it's both more and less interesting than you might expect from the headline. The short version is that retro tech YouTuber Bits und Bolts has been overclocking a whole lot of Socket 370 CPUs, and he's found a startling connection between the internal resistance of Pentium III processors and how well they overclock.

Specifically, what he found was that, within a specific stepping, processors with lower resistance between the VCC and Ground pins on the CPU typically achieve higher clock rates with less voltage when overclocked. "More MHz, fewer millivolts" is exactly what you want when overclocking, so if the correlation holds up, this is actually quite the discovery, at least for retro overclockers. So does it hold up?

Actually, yeah, mostly. There are a few caveats; he has at least one processor (out of dozens) that doesn't fit the pattern, and you can only compare resistance between processors of the same model (i.e. Celeron vs. Celeron) and on the same stepping; an A1 processor with higher resistance might still be a better overclocker than a B1 processor with lower resistance because it's relatively better for that stepping. He also found that processors which were binned higher by Intel typically had lower resistance, which supports the theory.

Does any of this apply to processors made in this decade? Probably not, for a few reasons. For one, modern processors are enormously more complicated internally; there are many parallel paths for power to take, and the internal voltage regulation circuitry is incredibly complex. Attempting to measure this on a current chip will give you a resistance value low enough that the resistance of your probes and the LGA pads starts to become a concern. And while the minuscule power levels used for resistance testing aren't going to damage your processor, you could easily short two pads or even damage a pad just enough to ruin its conductivity, destroying the CPU.

So saying, probably don't try this at home on modern chips. But if you have old Socket 370 hardware, especially if it's from the Tualatin generation, Bits und Bolts would love to see your data. Pull the CPU, whip out your multimeter, and generate some data; then, go post it in his comment section under the video below.

And if for some reason you don't have a digital multimeter, it's the most important tool in any electronics tech's arsenal. You can pick one up today for just $20, and you'll surprise yourself with how useful it can actually be for all kinds of handyman and repair jobs. Either of the ones linked above is good enough to get you started.

A standard $20 multimeter can help predict the overclocking potential of certain processors. This recent discovery was shared in a YouTube video titled "Can a Multimeter Predict Your Silicon Lottery Wins?" by a retro-tech content creator.

The method involves testing the chip's internal electrical resistance. However, the technique has significant limitations you should be aware of before trying it:

How does it work, and what are the limitations?

• The theory: Lower internal resistance generally translates to better conductivity and less heat generation under load, indicating a higher potential for reaching high frequencies (winning the "silicon lottery").

• Focus on older hardware: Initial tests showed a stronger correlation with older (retro) processors. Modern CPUs feature extremely complex, dynamic power management built directly into the silicon, which masks these figures.

• Limited control group: Resistance measurements are only valid when comparing identical CPUs—those sharing the exact same model, revision (stepping), and architecture, and ideally coming from the same fabrication plant (fab). A reading from one chip might mean something entirely different for another model.

• The "Path of Least Resistance": The multimeter measures the shortest, easiest electrical path within the circuit. If there is a defect or high resistance in an isolated area of ​​the chip (such as the L1 cache), the processor might still fail to overclock successfully, even if the multimeter shows an excellent overall result.

While this is a fascinating experiment for vintage hardware enthusiasts, diagnostic tools built into the BIOS and stress-testing software remain the reliable standard for determining the limits of your silicon in modern computers.

 

mundophone

TECH

ENISA: DDoS attacks accounted for more than half of the incidents recorded in Europe last year

In total, DDoS attacks represented 51.3% of the incidents recorded last year, according to the latest edition of the ENISA report on the threat landscape in the European Union. Meanwhile, unauthorized access to systems accounted for 39.5% of the total.

The 2026 ENISA Threat Landscape confirms that cyber dependencies expand the attack surface and require a new level of vigilance to effectively prevent and mitigate the impact of cyber incidents.

The cyber threat landscape of the European Union is still shaped by a combination of recurrent threats. 

Key highlights include:

-Ransomware remains the most short-term impactful type of incident. 

-Geopolitical developments still influence cyber activity affecting the EU with hacktivist-led DDoS campaigns targeting essential entities. 

-Public administration continues to be is the most targeted sector. 

-Organisations across the EU are likely to continue facing a combination of cybercrime, cyberespionage and hacktivist activity driven by geopolitical developments. 

-Emerging AI models are expected to be increasingly used to support malicious operations.

To shape our understanding of the cyber threat landscape and the dynamics at work, ENISA collected and analysed incidents and events observed from 1 January to 31 December 2025 for this new edition of the Threat Landscape. Those events were gathered from open sources, as well as anonymised information shared by EU Member States and through the ENISA Cyber Partnership Programme.

Public administration emerged as the most affected sector, accounting for 31.8% of incidents, and 73% of the affected organizations were entities classified as "essential" or "important" under the NIS2 directive.

According to ENISA, financially motivated attacks accounted for 29.3% of all recorded incidents, with ransomware standing out in this category. Ransomware attacks also feature on the list of threats affecting a wide range of sectors during the period under review—a list that includes data breaches, phishing, and fraud.

Regarding social engineering, phishing was present in 77.8% of incidents where this technique was identified, with attackers increasingly relying on "phishing kits" and specialized services to carry out campaigns.

ENISA also highlights the rise of threats such as ClickFix and tactics like smishing, as fraud schemes continue to exploit compromised credentials and identity theft, alongside other social engineering techniques.

Digital infrastructures in attackers' crosshairs...According to the report, attackers are increasingly exploiting organizations' digital infrastructures—including software vendors, third-party services, and cloud platforms—meaning a single incident can have consequences for a broader range of entities. Attacks on supply chains, third-party vendors, and other digital infrastructures continued to cause large-scale, high-impact incidents throughout 2025, the agency reports.

Alongside public administration, business services were also among the most affected sectors, accounting for 8.5% of the incidents recorded by ENISA. These were followed by transportation (8%), manufacturing (6.9%), and the finance and banking sector (5.6%). However, attack patterns and the impact of threats vary by sector. For instance, in the public administration sector, 82% of recorded incidents were ideologically motivated DDoS attacks, making the sector a primary target for campaigns linked to political and geopolitical events.

Geopolitical developments—including the ongoing war between Russia and Ukraine and the escalating conflict in the Middle East—also significantly impacted the threat landscape in 2025. Ideologically motivated operations accounted for 57.3% of observed incidents.

As detailed in the report, hacktivism cases were linked to campaigns involving DDoS attacks against public services, essential entities, and organizations connected to political events or the support of specific countries involved in conflicts.

During the period under review, 4,709 claims of hacktivist attacks against EU Member States were recorded. More than 89% of these involved DDoS attacks.

State-aligned threat actor groups also continued to conduct cyber-espionage operations, which accounted for 5.9% of observed incidents. On one hand, groups linked to Russia focused primarily on central government and diplomatic entities, while groups associated with China showed greater interest in the transport sector.

Vulnerabilities and AI-powered tools...Exploiting vulnerabilities remained a primary method for gaining unauthorized access to systems in 2025. Over the past year, more than 48,000 new vulnerabilities were recorded—a 22% increase compared to the previous year.

ENISA reports that these security flaws were the root cause of 60.4% of unauthorized access incidents where the attackers' entry method could be identified. These include both newly discovered vulnerabilities and known flaws that persist in systems that have not yet been updated.

AI is also gaining ground in cybercriminal operations, enabling faster, automated, and easier-to-execute attacks. The agency expects this trend to intensify, with the technology being used in an increasing number of attack stages and lowering the barrier to entry for attackers.

Looking ahead, ENISA anticipates that the major threats identified in 2025 will continue to impact European organizations. The realms of cybercrime, cyber-espionage, and hacktivism are expected to remain influenced by geopolitical developments. Cybercrime is projected to persist as a leading source of attacks and disruptions.

The agency also warns of two factors that could increase risks for organizations: growing reliance on external suppliers, services, and infrastructure, and the increasing use of AI.

mundophone

TECH

One of mathematics' greatest enigmas and AI

For decades, certain mathematical problems remained unsolved despite the efforts of experts worldwide. Now, artificial intelligence systems are venturing into this territory in ways that surprise even researchers accustomed to major breakthroughs. One company claims to have achieved dozens of new results in just a few weeks. Yet, while the speed is impressive, another question is gaining importance: how do we verify, understand, and transform these discoveries into mathematical knowledge?

In late August, OpenAI began internally training a model that has not yet been released to the public. Just 24 days later, the company made an extraordinary claim: the system had reportedly solved over 100 mathematical problems that had remained open for years.

These alleged results span various areas of mathematics. However, the case attracting the most attention involves a problem that has held a special place in the field for decades: Navier–Stokes.

OpenAI itself acknowledges that even its mathematicians did not anticipate such a rapid pace. According to the company, the model managed to produce results on a scale that raises a question beyond the mere ability to solve equations.

If an AI can generate potential solutions faster than experts can analyze them, the bottleneck shifts from simply finding an answer to verifying what has actually been discovered.

There is also a significant distinction between the claim regarding Navier–Stokes and the more than 100 other problems mentioned by the company. For Navier–Stokes, OpenAI published a manuscript and a formalization in Lean. For the other results, no complete list accompanied by public proofs has been presented so far.

This means that the figure of over 100 remains, for now, a claim made by the company itself rather than a set of results that has been independently examined by the mathematical community. The Navier–Stokes existence and regularity problem is one of the Clay Mathematics Institute's seven Millennium Prize Problems. Simply put, it seeks to determine whether the equations used to describe fluid motion in three dimensions can develop certain singularities in finite time.

OpenAI's model reportedly produced a solution indicating that this behavior can occur—and, more importantly, a formal proof in Lean.

This formalization is significant because Lean allows for the mechanical verification of a proof's logical steps. Even so, this does not mean the problem has been officially resolved.

The Clay Mathematics Institute acknowledged that the issue appeared to have been solved but noted that there is a specific process for recognizing a solution. The work must be published in an appropriate venue, remain available for scrutiny for at least two years, and gain general acceptance among mathematicians.

As this discussion unfolded, another issue arose: how should the community react to mathematical results produced by AI?

In September, 25 Fields Medalists and other mathematicians issued a statement criticizing how open problems were being used in the AI ​​race. The concern was not simply to deny the capabilities of these systems.

The point was different. In mathematics, solving a problem is only part of the process. It is also crucial to understand the idea behind the proof, explain why it works, acknowledge prior work, and enable other researchers to build upon that knowledge.

The company's response was announced on September 21: an advisory group comprising nine mathematicians began advising OpenAI on matters related to mathematics and artificial intelligence.

The group is affiliated with the Institute for Advanced Study in Princeton and includes prominent figures such as Timothy Gowers, Martin Hairer, Edward Witten, Ravi Vakil, and other distinguished researchers.

The group's role will be to help evaluate and communicate new results, recommend academic standards, and offer critical feedback to OpenAI itself when necessary. Its members will not be paid by the company and will be able to participate in determining the group's composition. But there is a clear boundary. The group lacks the power to dictate the pace of OpenAI’s internal research or to veto company decisions. Its role is to bridge the gap between AI labs and the mathematics community.

Mathematical enigmas and the controversy...These new achievements are part of a wave of AI results flooding the field of mathematics. Over the past year, AI has enabled major breakthroughs, leaving mathematicians grappling with rapid changes in their discipline. The new result, concerning one of mathematics' most important problems, represents "the spectacular culmination of the trajectory we’ve seen over the last 12 months," said OpenAI researcher Sébastien Bubeck during a press conference on September 8.

In a statement published on his website on September 7—alongside the solution regarding forced Euler flows—Buckmaster stated that his team's results mark a "Deep Blue-Kasparov moment," referencing the historic 1990s milestone when a supercomputer defeated the best human chess player. "The community needs to have a serious, unhurried discussion about the next steps," he said.

Mathematicians are certainly paying attention. Albritton learned of Buckmaster and Alpöge’s result around 1 a.m. while awake with his newborn baby. He stayed up until 6 a.m. discussing the matter with colleagues.

Despite the problem's mathematical significance, its solution will not have major practical implications, Eyink notes. The Navier-Stokes equations describe a fluid as a continuum, but real-world fluids are composed of individual molecules and atoms; thus, it is already known that there is a limit beyond which the equations cease to be valid. It is more a matter of prestige, says Eyink. "There is immense mathematical celebrity associated with these equations."

This also raises concerns regarding how credit for a discovery is distributed. After initial rumors began to circulate, Buckmaster says he held a series of discussions with OpenAI researchers about how to present the two results. In these discussions, he claims there was a request to exclude his co-author, Alpöge, who works for Anthropic, an OpenAI competitor. Buckmaster’s account also raises questions about whether AI agents had access to the progress made by Buckmaster and Alpöge. OpenAI denies that its AI agents had direct access but states: “while unlikely, we cannot rule out that anonymized data derived from their use of our products helped improve our models.”

The major implication of this breakthrough may be the issue it raises regarding the difficulty of assigning credit when AI is involved, says Eyink. "To me, this is the truly serious and ongoing problem and question that needs to be resolved."

This reveals the scale of the emerging problem.

The discussion is no longer just about whether an artificial intelligence can find a solution that no human had previously discovered. The question now becomes what happens when these machines generate potential discoveries faster than experts can verify, interpret, and incorporate them into existing knowledge.

And perhaps this is the true test of the new era of AI-driven mathematics: not just discovering more, but figuring out how to turn that speed into reliable science.

mundophone

 

NIKON

Nikon debuts Z5IIc full-frame mirrorless camera built for budget creators

Nikon has dropped the Z5 IIC, a revised viewfinder-less full-frame mirrorless over last year's Z5II with one clear mission: to give content creators something with the ease of smartphone photography and the power of higher-end digital imaging at an affordable price.

The Nikon Z5IIC is an entry-level full-frame camera with a 24.5MP BSI CMOS sensor. As its name implies, the sensor, along with many of the camera’s other specs and features, are borrowed from the company’s larger, more expensive Z5II; Nikon has simply crammed it all into a smaller body, leaving off a few extras like the EVF, high-resolution screen and second card slot.

There isn’t much new about the Z5IIC, as it’s essentially an existing camera with some features removed, rather than added. This means that the main change is its design, which we’ll mostly cover in the Body and handling section. However, as a high-level overview, the camera has a squared-off body with a grip style similar to that of the Z5II (which ends up making it feel a bit chunky in the hand).

This design, and the EVF it omits, means it occupies a new position in Nikon’s full-frame mirrorless lineup. While the video-focused ZR may appear to fit that description, the Z5IIC’s ergonomics and user experience are much closer to that of a super-sized Z30, but even more focused on stills and hybrid use. This is the first EVF-less mirrorless camera that Nikon has made aimed as much at photographers as videographers.

Beyond the design, Nikon has introduced one minor upgrade to the camera’s autofocus system: in addition to the 9 types of subject the Z5II was trained to recognize, the Z5IIC gains an “aquatic life” mode, mainly designed for taking pictures in aquariums. The camera also gains the highly-customizable grain effect the company introduced on the Zf, which is likely a bigger deal than it seems in the age of social media feeds dominated by simulations of film.

Unlike its fuller-featured sibling the Z5 II, there's no EVF here, which lends to a flat, lightweight chassis that can slip into carry bags and large jacket pockets. Offered in both retro silver and minimalist black, the magnesium-alloy body maintains dust- and drip-resistance while still being portable. Nixing the EVF also helped lower the cost enough to perhaps usher more buyers into the Z-mount ecosystem. As before, running the show is the same 24.5MP backside-illuminated (BSI) CMOS sensor and EXPEED 7 processor.

To cater to modern creators (or really, phone upgraders used to screen-based shooting), the Z5 IIC has intuitive, automated workflows. The camera is the first Nikon full-frame mirrorless to incorporate Scene modes right on the main control dial—all 15 of them, ranging from night landscapes and food to portraits and twilight. Whichever preset is selected, the camera will automatically optimizes exposure, white balance, and tone curves in the background. The 3.2-inch fully articulating LCD touchscreen offers tap to focus, double-tap to zoom, and swipe through photos, just like a phone.

Nikon’s deep-learning AI autofocus algorithms can also recognize 10 subject types, but this time, Nikon has gone beyond humans, vehicles, birds, and pets for its target database and included Fish into the algorithm. For low-light situations, the autofocus operates down to -10 EV, assisted by a 5-axis in-body image stabilization (IBIS) system with up to 7.5 stops of reduction compensation.

On the videography end of things, the Z5 IIC can do 4K/60 (with an APS-C/DX crop) and Full HD up to 120 fps for slow-motion capture. There's also support for 12-bit N-RAW recording, H.265 10-bit color, and N-Log profiles alongside a dedicated Product Review Mode that prioritizes immediate focus shifts to handheld items. Plug-and-play UVC/UAC compliance allows the unit to double as a high-definition webcam or streaming hub with a simple USB connection. 

Obviously, there are some potential negatives to the cost-cutting. For one, the aforementioned lack of EVF, which I can't live without for framing compositions. Storage is restricted to a single UHS-II SD card slot, mechanical shutter speed tops out at 1/4000th of a second, and the rear monitor, while functional for touch gestures, retains a modest resolution.

Priced at $1,399.95 for the body alone (or $1,699.95 bundled with the NIKKOR Z 24-50 mm f/4-6.3 kit lens), the Nikon Z5 IIC is slated to hit retail shelves in mid-October. 

 

by mundophone

 

TECH

Making AI more trustworthy: Path discovered to make AI models red-flag their doubtful answers

Artificial intelligence models can give users the wrong answer and do so with great confidence. They can also hedge and warn that they are unsure — even when they get the answer right.

A study led by UC Riverside computer scientists helps explain why. Researchers found that confidence and correctness can arise from different internal features within large language models, challenging the assumption that a model’s confidence is a reliable indication of whether its answer is accurate.

Their discovery could help build more reliable AI models. As large-language models are increasingly used to inform decisions and complete tasks, developers need better ways to determine when their answers can be trusted. 

By identifying internal features associated separately with confidence and correctness, the UCR-led research points toward ways AI systems could be adjusted so they are more confident when they are right and more cautious when they are likely to be wrong.

“The main assumption in the field is that when the model is confident, it is likely to be correct, and when the model is unsure, it is more likely to be incorrect,” said Het Patel, a UCR computer science doctoral student and lead author of the study. “But we often see the counterexamples that are well documented. A model can answer with certainty, but also be wrong, or it can answer while being less confident and can be correct.”

The researchers went beyond documenting that mismatch. They identified internal features associated with confidence and correctness and showed that altering some of them could change model behavior without the costly process of retraining an entire model.

Patel explained that modern AI models are created by training enormous networks of mathematical units on vast amounts of data. During training, the network repeatedly adjusts billions of numerical parameters, called weights, as it learns patterns in the data. In a language model, those learned patterns allow it to predict which words are likely to follow others and ultimately generate responses to questions.

Patel and his colleagues wanted to know what was happening inside the models when confidence and correctness did not match.

They studied two “open-weight” large-language models — Meta’s Llama-3.1-8B and Google’s Gemma-2-9B — whose internal workings can be examined by researchers. Using multiple-choice questions, they separated responses into four groups based on whether answers were correct or incorrect and whether the models were confident or uncertain.

They then used tools called “sparse autoencoders” to examine the models’ internal activity. That allowed them to determine which features became active with particular behaviors.

The analysis identified three kinds of features: those associated primarily with uncertainty, those associated primarily with incorrect answers, and “confounded” features associated with both. The researchers then suppressed selected features as the models answered questions. Patel compared the process to “turning knobs” inside a model to see how its behavior changed.

The differences were striking. Turning off features associated purely with uncertainty sharply reduced accuracy, suggesting those features play an important role in producing good answers. By contrast, suppressing most features associated solely with incorrect answers had little effect.

The confounded features produced a different result. Suppressing features associated with both uncertainty and incorrectness improved accuracy by up to 1.1% while reducing the models’ uncertainty by up to 75%. Similar effects appeared across different question-answering benchmarks.

The interventions were made when an already-trained model is answering questions, and do not require retraining it.

“In a sense, it’s kind of like adjusting or modifying the values of these activations or features after the fact to kind of get the behavior you want,” Patel said.

Another experiment suggests the internal signals could eventually help AI systems decide when not to answer. Using just three of the confounded features from a single middle layer of the Llama model, the researchers could predict whether the model was about to answer incorrectly. Having the model decline to answer the questions flagged this way raised its accuracy from 62% to 81% while it still answered about 53% of the questions.

By comparison, giving the model an “I don’t know” option and letting it abstain on its own raised accuracy only to about 64%. The results show the findings are not only observational; the same internal signal points to a practical step developers can take to make AI systems more reliable. 

The researchers also found evidence that these internal features were not tied narrowly to individual benchmarks. Features identified using one benchmark produced similar effects when applied to others, suggesting they reflected more general characteristics of the models.

Patel said the approach could extend beyond confidence and correctness. Researchers could search for internal features associated with other desirable or undesirable AI behaviors and test whether manipulating them changes how models perform.

“You could pick another behavior you want or don’t want, find the features related to it the same way, and then work on those internal features to drive that behavior or reduce it,” Patel said.

The study, “Are LLM Uncertainty and Correctness Encoded by the Same Features? A Functional Dissociation via Sparse Autoencoders,” has been accepted for presentation at the SIAM International Conference on Data Mining in Salt Lake City in November.

In addition to Patel, the co-authors are UCR Professor Evangelos E. Papalexakis, the Ross Family Endowed Term Chair in Computer Science; Jia Chen, a UCR associate professor of teaching; and Arizona State University researchers Tiejin Chen and Hua Wei.

UC Riverside computer scientists