Learn more about Uplevel's ML Issue Classification
Overview
Uplevel’s Time Allocation ML Issue Classification provides key visibility into engineering effort and time spent on New Feature Development vs. Defects vs. Sustenance without dependencies on how teams are tagging work. Learn more by visiting this article on how ML Issue Classification Works.
- Defect - This category contains issues for work related to bugs and defects.
- New value creation - This category contains issues for work related to new features and enhancements.
- Sustenance - This category contains issues for work related to operational efficiency and improvements to maintain and improve reliability and safety. Some types of work included are KTLO, maintenance, and tech debt.
Details
- Uplevel first classifies issues as defect or not defect. Uplevel then classifies the non-defect issues as either new value creation or sustenance.
- Our model uses standard issue fields like issue type, summary, description, and assignee information, so a high level of hygiene is not required. It does use epic if one is linked but it’s not problem if there isn’t an epic.
- Uplevel utilizes a default confidence percentage of 60% to categorize the issues. If below 60% confidence for a specific issue, Uplevel will aggregate the work under “Not linked to a ML Issue Classification”. This confidence percentage is configurable.
Accuracy / Results
Uplevel has seen great success in the accuracy of our ML Issue Classification model with current customers to-date. We’ve directly compared our ML Issue Classification models to custom aggregations and manually tagged categories already leveraged by Uplevel customers today, which has provided the following results:
- The current model for defect vs not defect:
- Overall ~93% (87%-96%) of issues have the predicted classification matching the manual classification. Looking at allocation time it is ~93% (86%-99%) matching.
- The current model for new value creation vs sustenance:
- Overall ~70% (64%-77%) of issues have the predicted classification matching the manual classification. Looking at allocation time it is ~71% (69%-76%) matching.
FAQ
- Is this an LLM?
- No, it is not an LLM. It is an ML (machine learning) model.
- What if I have a very specific definition for defect, new value creation, or sustenance that I want to use instead?
- We can apply a rule based on your definition of that category that you can see in an allocation scheme.
- We have plans to allow overrides to the model value.
- What if I want to add a category (e.g. Compliance)?
- We can apply a rule based on your definition of that category that you can see in an allocation scheme.
- We cannot add a category to the model output.
- What if I think the classification is wrong for an issue / epic?
- We can re-classify issues manually today and have plans to allow customers to re-classify issues themselves in the short term.
- What is the default confidence percentage required to classify issues vs. reporting as “Not linked to a ML Issue Classification”?
- Uplevel utilizes a default confidence percentage of 60% to categorize the issues. If below 60% confidence for a specific issue, Uplevel will aggregate the work under “Not linked to a ML Issue Classification”. This confidence percentage is configurable.