Problems to be solved
Open problems where help is welcome.
Digital twin
Faster simulation. Build the digital twin to do more realistic simulations faster.
Example: changing a stigmator and acquiring an image takes about 0.13 s at 512 px in the live twin. The more realistic emtwin path, ray tracing through the column followed by abTEM multislice, is optional. If it ran at close to that speed, an automated alignment of many steps could use the realistic model at every step.
GUI controlling
- Automated alignment through the GUI.
Workflow design
- Find an area the user describes. For example, find an MgO cube of 10 nm size, or a carbon nanotube with a diameter of 3 nm.
- Find a feature and its zone axis. For example, find an MgO nanotube of 20 nm, then find its zone axis.
Harness design
- Record a skill and reuse it robustly. Record what happens on the digital twin and turn it into a skill, with
safety checks and a good architecture.
- How do we capture the steps, parameters and checks from a run that worked?
- How does a skill still work when the starting state is different?
- How is a skill tested on the twin before it runs on a microscope?
- Use the Wiki to distill conversations and manage memory. Conversations and runs are stored in the
Wiki today, but nothing turns them into short, lasting memories.
- How do we extract what is worth keeping from a long conversation?
- How do we keep memories current, and retire the ones that are out of date?
- How does a personal lesson get reviewed and shared with everyone?
- Manage knowledge by instrument. Wiki libraries are organised by person today, not by instrument.
- How do we keep each instrument's calibration, habits and known problems together?
- How does the assistant pick the right memories and skills for the instrument in use?
- How do we keep what was learned on the twin apart from what was learned on real hardware?
- Make the harness as good to use as Claude Code, and right for a microscope lab.
- How does the assistant show its plan, progress and tool calls clearly while it works?
- How does it stay on track through a long session?
- Actions on an instrument cannot be undone like a code change. When should it stop and ask?
- How do images, spectra and live instrument state become part of the conversation?