Product Enhancements:

• Fixed additional vulnerabilities found in v2.0.521

Product Enhancements:

• When toxicity is detected, Shield will return subcategories of toxicity or toxicity violation types (i.e. Profanity, Harmful Request, Toxic Content - which covers hate speech and other discriminative languages)
• Users now have the ability to add “hints”, a string descriptor of the type of sensitive data to be caught when configuring the sensitive data rule. This will provide the LLM with additional information on what to look for in the text it is evaluating leading to improved performance. Refer to the rule configuration guide for more detail.
• The product now supports the use of self signed SSL certificates.

Bug Fixes:

• The embedding model configuration that’s only used when Arthur Chat is enabled was required when Arthur Chat was disabled in v2.0.495
• Fixed an issue with user role migrations in Shield v2.0.495 upgrade

Notes:

• Rolled back the recently released code classifier and are currently making improvements based on insights with new datasets.

Product Enhancements:

• Added a code classifier to the Prompt Injection rule to detect and skip evaluation of code to reduce false positives
• The toxicity rule check’s max token limit configuration is now exposed on the installers
• A new endpoint for resetting user password is added. The new attribute was also added to the user creation endpoint that provides the control to indicate whether the password must be changed at the first login.
• Refactored the pre-defined UI access roles with TASK-ADMIN and CHAT-USER roles

Bug Fixes:

• Improved request input validation to enhance robustness of Shield working with unexpected characters, such as the null character.
• Fixed the issue with duplicate logging in the application log

New Features:

• OpenAI has been added as an LLM service provider option in addition to Azure for the Shield rules that leverage LLMs (ex: Hallucination, Sensitive Data)
• Our most recent experimental hallucination rule has been now promoted as the Hallucination V3 rule in beta mode. Benefits of V3 include: Increased speed & decreased cost compared to V2, Improved labeling through algorithmically optimized language models that require fewer few-shot examples for training

Product Enhancements:

• The claim classifier in the hallucination rule has been retrained with more data to improve the detection of text chunks that should be skipped for evaluation as claims
• Improved the stability of the toxicity rule executions. A circuit breaker has been introduced to skip the rule evaluation if the number of tokens exceeds the default value of 1,200 to reduce the number of requests that fail due to latency.
• A new endpoint was introduced in Arthur Chat that returns the list of most recent conversation IDs

Bug Fixes:

• Fixed an issue where claims that had a negative order value were being shown first in the UI even when they were at the bottom of the overall message

Product Enhancements:

• The hallucination claims processor now better handles context with formatted lists
• The experimental hallucination rule received another major upgrade. It is now in beta.
• Users can now view details for keyword and regex rules in the inference deep dive UI
• Users can now filter on a single column by multiple values in the UI inference deep dive UI
• Users can now sort by timestamp in the UI inference deep dive UI
• In the inference deep dive UI, the hallucination claims are ordered to align with the generated response for new inferences moving forward
• Users will now see a message in the “view details” section on an expanded inference if there is additional information available about validation status

Bug Fixes:

• Fixed an issue where the hallucination rule was not considering markdown context like URL links

Product Enhancements:

• Users now have the option to add “Task Name” as a column to the inference deep dive table in the Admin UI
• Users can now search by task name using the search bar on the Admin UI task page
• Added the ability to filter by Inference ID on both the API and the Admin UI

Upgrade Requirement:

• When performing an upgrade, you must have Arthur Shield v2.0.356 on your existing instance before running the upgrade

New features:

• Enhanced the Admin UI Inference Deep Dive page with the ability to explore all hallucination results details including the claims and the context
• Added the ability to collect and retrieve user feedback on the inference context and the Shield results using the /api/v2/feedback endpoints
• Added the ability to capture user_id with inferences on the validation endpoints

Product Enhancements:

• Improved the text parser in the Hallucination rule for handling common abbreviations and markdown
• The LLM backed Shield rules will now return more informative errors when something goes wrong with LLM interactions

Bug fixes:

• Fixed an issue in the UI where the page refresh would trigger after a short period of time even though the user’s session is still alive

Deprecation:

• We are deprecating the ability to set threshold on the PII confidence score. The configuration has been removed from the UI and it is deprecated on the API.

Product Enhancements:

• Improved the concurrency of the toxicity rule execution for performance improvement
• Reduced the false positives of partial profanities, accidental profanities, and obscured niche scandalous internet terms in the toxicity rule
• Added the capability for the hallucination v2 rule to return partial results instead of responding with “Unavailable” for the whole inference when it’s not able to successfully evaluate all claims
• Retrained the prompt injection model and updated the input truncation schema for higher accuracy
• Introduced a new experimental version of hallucination rule (not ready for production use)
• The hallucination v1 rule has been deprecated
• UI: It’s now more clear that sorting the inference deep dive table is done according to timestamp
• UI: Added the ability to navigate to specific pages on the tasks page
• UI: Made usability improvements to filtering on the inference deep dive page

Bug Fix:

• Fixed an issue where an inference was labeled as an overall pass but there was at least one rule that failed

New Features:

• Single-sign-on (SSO) authentication with customer IdPs over OpenId Connect (OIDC)

Product Enhancements:

• The toxicity classifier model was retrained with an upgraded harmful request dataset with better false examples including innocent requests
• All PII configurations are exposed in the UI for creating, editing, and viewing the task rule
• The document selection of the Chat UI will now persist for the entirety of a user’s session

Bug Fixes:

• Fixed an issue where refreshing while on the task page in the shield admin UI would bring the user to the inference deep dive page
• In version 2.0.293, the prompt injection rule was not able to evaluate prompts larger than 512 tokens

Product Enhancements:

• Improved the prompt injections rule with a focus on jailbreaking and instruction manipulation
• The latency metric in milliseconds is captured for each Shield rule execution