OpenTelemetry (often shortened to OTel) is an open-source project that gives you vendor-neutral APIs, SDKs, a wire protocol called OTLP and a Collector for traces, metrics and logs. It does not store or display anything. You point it at a backend of your choice. The practical gain is that you instrument once and can later change backends by changing configuration.
Maturity differs by language and signal, so check the status tables on opentelemetry.io for your stack.
Step 1: A service to instrument
Create a small service using only the Python standard library, plus one OpenTelemetry package for the server side. It has an endpoint that calls another HTTP endpoint, which gives us a parent-child relationship to look at.
# app.py
import json
import random
import time
from socketserver import ThreadingMixIn
from urllib.request import urlopen
from wsgiref.simple_server import WSGIServer, make_server
from opentelemetry.instrumentation.wsgi import OpenTelemetryMiddleware
class ThreadingWSGIServer(ThreadingMixIn, WSGIServer):
daemon_threads = True
def respond(start_response, payload):
start_response("200 OK", [("Content-Type", "application/json")])
return [json.dumps(payload).encode()]
def app(environ, start_response):
sku = environ["PATH_INFO"].rsplit("/", 1)[-1]
if environ["PATH_INFO"].startswith("/price/"):
time.sleep(random.uniform(0.01, 0.05))
return respond(start_response, {"sku": sku, "price": 42})
with urlopen(f"http://localhost:8000/price/{sku}", timeout=2) as response:
quote = json.load(response)
return respond(start_response, {"status": "ok", "quote": quote})
if __name__ == "__main__":
make_server("", 8000, OpenTelemetryMiddleware(app), server_class=ThreadingWSGIServer).serve_forever()
Set up a virtual environment:
python3 -m venv .venv
source .venv/bin/activate
pip install opentelemetry-instrumentation-wsgi
Step 2: Add zero-code instrumentation
The opentelemetry-distro package gives you a launcher that wraps your program.
The opentelemetry-bootstrap command then detects your installed libraries and
installs matching instrumentation packages. With a mainstream web framework, that alone
produces server spans. The standard library's server has no such hook, which is why the code
above wraps the app in the WSGI middleware; the outgoing urllib call is
instrumented with no code at all.
pip install opentelemetry-distro opentelemetry-exporter-otlp-proto-http
opentelemetry-bootstrap -a install
Run with console output first, so you can see what is produced without any backend:
export OTEL_SERVICE_NAME=order-demo
export OTEL_TRACES_EXPORTER=console
export OTEL_METRICS_EXPORTER=none
export OTEL_LOGS_EXPORTER=none
opentelemetry-instrument python app.py
In another terminal:
curl http://localhost:8000/order/abc
You should see JSON spans printed in the service's terminal. Look for three of them: a
server span for /order/abc, a client span for the outgoing urllib
call and a server span for /price/abc. All three share a trace_id,
and each parent_id points at its caller. If they do, you have working
propagation, even though this example happens to call itself.
Step 3: Send to a real backend over OTLP
OTLP can run over gRPC (default port 4317) or HTTP (default port 4318). HTTP is usually
easier to get through proxies and firewalls, and it is what we will use here. When you set
the base endpoint variable, the SDK appends /v1/traces for traces.
export OTEL_SERVICE_NAME=order-demo
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
export OTEL_EXPORTER_OTLP_COMPRESSION=gzip
export OTEL_RESOURCE_ATTRIBUTES=service.version=1.0.0,deployment.environment.name=staging
export OTEL_METRICS_EXPORTER=none
export OTEL_LOGS_EXPORTER=none
opentelemetry-instrument python app.py
If your backend needs an auth token,
OTEL_EXPORTER_OTLP_HEADERS=authorization=Bearer%20<token> sets the
header. Values must be URL-encoded, hence %20. Keep tokens in a secret, not in
a Dockerfile.
These variables come from the OpenTelemetry specification and work across most language SDKs, which is why they are worth learning:
| Variable | Purpose |
|---|---|
OTEL_SERVICE_NAME |
Sets service.name, the most important attribute you will set |
OTEL_RESOURCE_ATTRIBUTES |
Comma-separated key=value pairs that describe the service |
OTEL_EXPORTER_OTLP_ENDPOINT |
Base URL for the OTLP exporter |
OTEL_EXPORTER_OTLP_PROTOCOL |
grpc, http/protobuf or http/json; SDK support varies |
OTEL_EXPORTER_OTLP_HEADERS |
Headers for auth |
OTEL_TRACES_SAMPLER |
For example parentbased_traceidratio |
OTEL_TRACES_SAMPLER_ARG |
The sampler's argument, for example 0.25 |
OTEL_PROPAGATORS |
Defaults to tracecontext,baggage |
OTEL_SDK_DISABLED |
Set to true to turn the SDK off |
Step 4: Add a manual span where it counts
Automatic spans cover the framework edges. They do not know that your code applies a discount. Add spans with the API only where your team needs that view.
from opentelemetry import trace
from opentelemetry.trace import Status, StatusCode
tracer = trace.get_tracer("order-demo.pricing")
def apply_discount(order_total, customer_tier):
with tracer.start_as_current_span("apply-discount") as span:
span.set_attribute("app.customer.tier", customer_tier)
try:
return calculate_discount(order_total, customer_tier)
except ValueError as error:
span.record_exception(error)
span.set_status(Status(StatusCode.ERROR))
raise
The opentelemetry-instrument launcher configures the SDK for you, and this code
only uses the API. Libraries should depend on the API alone, too: without a configured SDK,
the API does nothing, at almost no cost.
Some habits to keep from the start:
-
Name spans for the operation, with low cardinality:
apply-discount, notapply-discount-customer-88213. IDs go in attributes. -
Use semantic convention names when one exists, such as
http.request.method,http.response.status_code,url.pathandhttp.route. Use your own prefix, such asapp., for business attributes. - Don't put personal data in attributes unless you have a reason and a policy for it.
- Don't wrap hot loops in spans.
Step 5: The same idea in other languages
The zero-code approach exists for other languages too. The commands differ, but the environment variables stay the same.
Java. Download the agent JAR from the OpenTelemetry Java instrumentation releases and attach it:
export OTEL_SERVICE_NAME=order-api
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
java -javaagent:./opentelemetry-javaagent.jar -jar order-api.jar
Node.js. Install the auto-instrumentation meta package and load it before your application:
npm install --save @opentelemetry/api @opentelemetry/auto-instrumentations-node
export OTEL_SERVICE_NAME=order-web
export OTEL_TRACES_EXPORTER=otlp
export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318
node --require @opentelemetry/auto-instrumentations-node/register app.js
.NET. OpenTelemetry .NET Automatic Instrumentation installs through a script from its releases page, which sets the profiler environment variables. On Kubernetes, the operator can do this for you, as explained in the operator guide.
Step 6: Fix what you will probably hit first
| Symptom | Likely cause | Fix |
|---|---|---|
| No spans at all | Wrong endpoint or a blocked port | Switch temporarily to the console exporter to separate SDK from network problems |
| HTTP 404 from the exporter | A signal path doubled up or missing | With OTEL_EXPORTER_OTLP_ENDPOINT, give only the base URL. OTEL_EXPORTER_OTLP_TRACES_ENDPOINT needs the full path |
| gRPC exporter hitting port 4318 | Protocol and port do not match | Use 4317 for gRPC and 4318 for HTTP |
Service appears as unknown_service |
service.name is not set |
Set OTEL_SERVICE_NAME |
| Separate traces for each service | Lost propagation, or differing propagators | Check that traceparent arrives at the next hop and set the same OTEL_PROPAGATORS |
| Spans disappear on short jobs | Batch exporter did not flush on exit | Shut down the provider cleanly at exit |
| Missing spans from a library | Instrumentation package not installed | Re-run bootstrap after adding dependencies |
| Forked workers lose spans | Pre-fork application servers | Check the OpenTelemetry Python docs on working with fork process models |
Step 7: What to do next
- Add a Collector. Sending straight from the SDK is fine for learning. In production, a Collector gives you a place for batching, retries, filtering and routing. See running the Collector on Kubernetes.
- Decide on sampling. Start at 100% in staging. In production, make your decision from real volume. Tail-based sampling covers the options.
- Connect logs. Add the trace ID to your log format so that logs and traces connect.
-
Standardise attributes. Agree with other teams on
service.name,service.namespaceand environment values, because these are painful to fix later.
Sending to Wrentrix
Wrentrix accepts OTLP traces and logs over HTTP, in protobuf or JSON and with gzip, so the variables above apply directly. It works alongside existing collectors. The distributed tracing page describes what you get from that data. OTLP over gRPC and OTLP metrics are not part of what we list here, so ask us if you depend on them.
Frequently asked questions
Is OpenTelemetry a monitoring tool?
No. It is a set of APIs, SDKs and a Collector for producing and shipping telemetry, plus the OTLP protocol. You still need a backend to store, query and alert on the data. That separation is what makes the instrumentation portable.
Should I use OTLP over gRPC or HTTP?
Both are fine for most systems. HTTP on port 4318 tends to pass more easily through proxies and load balancers, and gRPC on port 4317 can be somewhat more efficient on persistent connections. Check what your backend or collector accepts, then use the protocol it supports.
What is the difference between the OpenTelemetry API and SDK?
The API defines the calls that your code and libraries make, and does nothing by itself. The SDK is the implementation that samples, processes and exports the data. Libraries depend on the API only, while an application configures the SDK or lets an agent do it.
Can I use OpenTelemetry without changing my code?
Yes, for many frameworks in Java, .NET, Python, Node.js and others. A launcher, agent or Kubernetes operator instruments common libraries automatically. You will probably add some manual spans later for business operations that a framework can't see.
Do I need an OpenTelemetry Collector?
Not to start, since SDKs can export directly to a backend that speaks OTLP. In production, most teams add one for batching, retries, enrichment, filtering and sampling. It also keeps backend credentials and routing out of your application configuration.