You copy a YAML snippet from documentation, a ticket, or a colleague’s message. The field names look correct. The reviewer approves it. CI passes. Then the deployment breaks - silently - because a key contains a character that renders invisibly but changes what the parser reads.
YAML is a text format parsed by machines and reviewed by humans. When invisible Unicode characters enter the picture, the two don’t see the same thing.
Index
- The problem
- Why this happens
- The most dangerous Unicode characters
- Defending your CI/CD pipelines
- A reasonable Unicode policy
- Using yuc
- Final thoughts
The problem
A simple example
A zero-width space (U+200B) has no visible rendering. Copy-paste from browsers, rich-text editors, or documentation tools can introduce one silently - it leaves no visible mark on screen.
Embed one inside a YAML key:
apiVersion: v1
kind: ConfigMap
The key looks like apiVersion. The parser reads apiVersion - a different string with the zero-width space appended. If your Go struct maps yaml:"apiVersion":
type Config struct {
APIVersion string `yaml:"apiVersion"`
}
unmarshaling silently produces an empty value:
Config{
APIVersion: "",
}
The field is just gone.
Homoglyphs
Unicode also enables homoglyph attacks - visually similar characters from different scripts.
| Character | Unicode | Script |
|---|---|---|
a | U+0061 | Latin |
а | U+0430 | Cyrillic |
This YAML:
аpiVersion: v1
contains a Cyrillic а, not a Latin a.
Humans may never notice the difference.
Why this happens
Unicode and UTF-8 are related but different concepts.
| Concept | Meaning |
|---|---|
| Unicode | abstract character set |
| UTF-8 | byte encoding of Unicode |
Most risky YAML files are:
- perfectly valid UTF-8
- but semantically dangerous Unicode
Humans review visually, and parsers compare exact characters.
That mismatch creates room for confusion and spoofing.
The risky Unicode characters
Invisible formatting characters
| Character | Unicode | Risk |
|---|---|---|
| Zero-width space | U+200B | invisible key corruption |
| Zero-width joiner | U+200D | rendering tricks |
| BOM | U+FEFF | parser inconsistencies |
| RTL override | U+202E | visual spoofing |
Confusing whitespace
| Character | Unicode |
|---|---|
| Non-breaking space | U+00A0 |
| Thin space | U+2009 |
These are especially dangerous in YAML indentation.
Defending your CI/CD pipelines
A layered approach works best:
- lint YAML
- scan for Unicode
- use strict unmarshaling
- validate schemas
Lint YAML
---
extends: default
rules:
document-start: enable
key-duplicates: enable
trailing-spaces: enable
Scan for Unicode
grep -rPn '[^\x00-\x7F]' . --include='*.yaml'
Use strict unmarshaling
decoder.KnownFields(true)
Validate schemas
Validate against:
- OpenAPI
- JSON Schema
A reasonable Unicode policy
Not all Unicode is dangerous.
For example, these are legitimate:
serviceName: 東京-api
description: 宮本武蔵のAPI
A practical policy:
| Area | Policy |
|---|---|
| YAML keys | ASCII-only |
| Identifiers | restricted |
| Values/comments | Unicode allowed |
Using yuc
Alternatively, you could use yuc - a configurable tool for checking your yaml.
This lets you define the severity, and rules for checking unicode in your yaml.
Install
macOS (Homebrew):
brew tap harshadixit12/yuc --force-auto-update
brew trust --formula harshadixit12/yuc/yuc
brew install yuc
Other platforms:
curl -fsSL https://raw.githubusercontent.com/harshadixit12/yuc/refs/heads/main/install.sh | bash
Source Repository:
https://github.com/harshadixit12/yuc
Example:
printf $'apiVersion\u200b: v1\nkind: ConfigMap\n' > bad.yaml
yuc bad.yaml