Microsoft has been making Computer use available within Copilot Studio for a few hours now.
“The Computer use allows agents to interact with websites and desktop apps by clicking buttons, selecting menus, and typing in on-screen fields. This allows agents to manage tasks even when an API is not available to connect directly to the system. If a person can use the app, the agent can too.”
What are the benefits?
- Adaptability related to changes in apps and websites.
- Real-time adjustment of screen changes.
- Compliance with organizational and industry standards.
How does it work?
The process is carried out on infrastructure hosted by Microsoft, allowing organizations not to manage their own servers.
UI automation in Copilot Studio uses advanced artificial intelligence algorithms to understand and replicate user actions. Whether it’s navigating between different applications, entering data into forms, or performing complex analyses, Copilot Studio can handle it all with a simple command. Users can create custom workflows that adapt to their specific needs, making the automation process highly flexible and customizable.
Scenarios:
- Automated data entry.
- Market research.
- Invoice processing.
Conclusions
Those familiar with the Power Platform know that there is a tool called Power Automate Desktop that partly embraces these themes. However, here we are in a very different playing field.
Adaptability and dynamism make our Agents much more performant, even with complex user interfaces.
Boom Done 💣
Follow me:
BEYOND THE PLATFORMS 
Leave a comment