Is CoPilot Ready? A Realistic Look at Current Functionality

As many organizations contemplate purchasing Copilot licenses and look to weigh the cost benefit of the investment…many find themselves asking: Is Copilot truly ready?

The appeal of Copilot is its potential to transform the way we work. By integrating into the applications we use every day, analyzing data across them, and identifying context and patterns – Copilot should be able to make automated suggestions, generate content and summarize vast amounts of data quickly and easily. This capability would make every worker more efficient and empower teams to tackle complex projects with greater efficacy.

Unfortunately, despite the potential of Copilot, its current functionality leaves much to be desired for many organizations. While the concept of an AI-driven assistant analyzing data, gleaning insights and automating tasks sounds appealing, the practical applications of Copilot in its current state are limited.

Some examples of current shortcomings:

• Data analysis in Excel is limited to less than 10,000 rows.
• Insights/Analysis of raw data seems limited to chart generation and very basic summarization.
• Content creation of Copilot is limited as it lacks the general knowledge of other LLMs like ChatGPT. By design Copilot is intended to have security in mind and limits its knowledge/capabilities to what it has found across OneDrive and SharePoint. While it tends to find and index data well it is inherently limited and often unable to pull in outside concepts or rely on common knowledge to generate content or analyze data.
• In many cases Copilot can scan and index data too well, presenting sensitive or proprietary data within its content/analysis. Not necessarily a short coming of the tool but identifying gaps in access and identify management within an organization.

At this point there are only two truly viable features that Copilot offers that perform well and offer efficiency:

1. Meeting Assistant – Summarizing Teams meetings from transcription

• Copilot for Teams does a decent job of this task and is able to summarize the content of hours of meetings into concise notes and action items.

2. Github Copilot – Released more than 2 years ago this Copilot acts as an AI-powered code completion tool.

• The oldest “copilot”, can streamline the coding processes and can boost developer productivity.

Unfortunately, while both use cases can provide value it may not justify the expense of the license for many organizations. While AI technologies hold immense promise, they often encounter hurdles in real-world implementation. In the case of Copilot, the gap between expectation and reality highlights the need for a more nuanced assessment of AI solutions before pursuing widespread adoption.

To be fair, Copilot is still in its nascent stages, and it’s reasonable to expect improvements and expansions in its functionality over time. However, for organizations weighing the investment in Copilot against its current utility, it is difficult to justify nearly double the cost of a standard E3 license for these limited features. Until Copilot evolves to address a broader range of use cases and deliver tangible value proportionate to its cost, your IT dollars are likely better spent elsewhere.

Instead, organizations may want to focus their funds and efforts on preparing for the AI future.

• Perform a policy and access audit
• Eliminate local storage and force users to OneDrive/SharePoint
• Pursue workflow automation utilizing Intune/Autopilot
• Perform a Copilot readiness assessment
  • Identify a cross functional team for a pilot
  • Identify specific use cases and teams that would benefit the most from Copilot
  • Explore other AI solutions and weigh ROI (i.e. Adobe AI, ChatGPT, etc.)