Langfuse Academy - Certificate

💡

Test your knowledge by answering the questions below. You need to score at least 8 out of 12 to pass.

1. (Module 1) Which statement BEST captures the primary objective of LLMOps?

A) To design new language models from scratch

B) To provide the operational practices that turn LLM prototypes into reliable, production-grade applications

C) To compress training datasets for faster pre-training

D) To fine-tune models purely for academic benchmarks

2. (Module 1) Which of the following challenges is NOT typically addressed by LLMOps?

A) Cost & latency monitoring

B) Prompt version management

C) Real-time GPU over-clocking

D) Evaluation & benchmarking

3. (Module 2) An LLM decomposes an open-ended goal into sub-tasks, delegates them to worker agents, and re-evaluates if needed. Which architectural pattern does this describe?

A) Tool Use (Function Calling)

B) Reflection (Evaluator-Optimiser)

C) Planning (Orchestrator-Workers)

D) Prompt Chaining

4. (Module 2) As you move from deterministic workflows toward multi-agent collaboration, which attribute increases the MOST while predictability generally decreases?

A) Deterministic execution

B) System agency / autonomy

C) Control over latency

D) Ease of unit testing

5. (Module 3) Within a trace, which observation type extends span semantics with model-specific attributes such as prompt, completion, and token usage?

A) Event

B) Span

C) Generation

D) Metric

6. (Module 3) Recording the p99 latency at the LLM layer primarily helps answer which of the following questions?

A) Which sessions hit the cost guard-rail?

B) Do we observe long-tail latency spikes in a particular region or release?

C) Which prompt version produced this output?

D) Did the router misclassify the user intent?

7. (Module 4) In the continuous evaluation loop, what is the step that follows the creation or curation of test datasets?

A) Deploy changes directly to production

B) Run experiments, then evaluate the results

C) Collect implicit user feedback

D) Red-team the application

8. (Module 4) Which of the following is an example of implicit user feedback that can be used for online evaluation?

A) Thumbs-up / thumbs-down rating

B) User dwell-time on the generated answer

C) Manual expert annotation

D) Model-based toxicity score

9. (Module 5) Which prompting strategy intentionally includes 1-5 example interactions to guide the model's style or structure?

A) Zero-shot

B) Few-shot / In-context

C) Chain-of-thought

D) Retrieval-Augmented Generation

10. (Module 5) What is the PRIMARY reason for versioning prompts in production environments?

A) To reduce token usage through compression

B) To enable instant rollback and traceability when quality or compliance issues arise

C) To automatically fine-tune a new base model

D) To avoid adjusting temperature settings

11. (Module 4) Why must automated evaluators powered by smaller LLMs be periodically calibrated against human-annotated samples?

A) They are deterministic and never change

B) They can drift or inherit biases, leading to mis-aligned scoring

C) Human annotations are required by law

D) Calibration lowers computational cost

12. (Module 3) In a multi-step LLM pipeline, which layer is MOST likely to dominate end-to-end latency if not instrumented properly?

A) LLM token generation

B) External tool calls such as database or vector store look-ups

C) Logging of user feedback

D) Version control operations

🎓 Langfuse Academy Certificate