feat(litellm): Gate prompt/response collection on data_collection option - #7180
Conversation
Modify the LiteLLM integration to respect the data_collection config for controlling whether chat messages, embeddings inputs, and response text are captured in spans. When data collection is enabled, the new gen_ai.inputs and gen_ai.outputs flags control what data is collected. When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility. Refs PY-2588 Refs #6748
Codecov Results 📊✅ 112607 passed | ⏭️ 6759 skipped | Total: 119366 | Pass Rate: 94.34% | Execution Time: 413m 35s 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 100.00%. Project has 2485 uncovered lines. Coverage diff@@ Coverage Diff @@
## main #PR +/-##
==========================================
- Coverage 90.21% 90.20% -0.01%
==========================================
Files 193 193 —
Lines 25361 25370 +9
Branches 9302 9310 +8
==========================================
+ Hits 22880 22885 +5
- Misses 2481 2485 +4
- Partials 1436 1432 -4Generated by Codecov Action |
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 78db0d6. Configure here.
| # Record input/messages if allowed | ||
| if should_send_default_pii() and integration.include_prompts: | ||
| record_inputs = False | ||
| if has_data_collection_enabled(client.options): |
There was a problem hiding this comment.
This might've already been discussed somewhere, in that case disregard, but: Should we also take integration.include_prompts into account here, so that users have more fine-grained control over which prompts will be sent?
There was a problem hiding this comment.
We've agreed that niche use cases like filtering by integration should be done in before_send().
The option will be deprecated once data collection has fully landed.
3 participants
Modify the LiteLLM integration to respect the data_collection config for controlling whether chat messages, embeddings inputs, and response text are captured in spans.
When data collection is enabled, the new gen_ai.inputs and gen_ai.outputs flags control what data is collected. When data collection is not configured, falls back to legacy send_default_pii and include_prompts settings for compatibility.
Refs PY-2588
Refs #6748