Encoding — Base64, hex, text encodings, URL, HTML

Summary#

Three small encoding surfaces, spelled exactly as in C#, all pure (no capability, no using required):

var b64  = Convert.ToBase64String(bytes);          // binary  → base64 text
var back = Convert.FromBase64String(b64);          // base64  → binary
var q    = Uri.EscapeDataString("a b&c");          // "a%20b%26c" — safe in a URL
var safe = WebUtility.HtmlEncode("<b>hi</b>");      // "&lt;b&gt;hi&lt;/b&gt;" — safe in HTML text

Signature#

string  Convert.ToBase64String(byte[] bytes)     // Binary → base64 text
byte[]  Convert.FromBase64String(string s)       // base64 text → Binary
string  Convert.ToHexString(byte[] bytes)        // Binary → hex text, UPPER case ("4F4B")
string  Convert.ToHexStringLower(byte[] bytes)   // Binary → hex text, lower case ("4f4b")
byte[]  Convert.FromHexString(string s)          // hex text (either case) → Binary
byte[]  Encoding.UTF8.GetBytes(string s)         // text → bytes in an encoding (also ASCII, Latin1, Unicode, UTF32)
string  Encoding.UTF8.GetString(byte[] bytes)    // bytes in an encoding → text
int     Encoding.UTF8.GetByteCount(string s)     // how many bytes the text takes
byte[]  Encoding.GetEncoding(437).GetBytes(s)    // a legacy code page, by number or name
Encoding enc = Encoding.UTF8;                    // an encoding is a value: a local, a field, a parameter
string  enc.WebName                              // "utf-8" — also EncodingName, CodePage, IsSingleByte, GetPreamble()
string  Uri.EscapeDataString(string s)           // percent-encode for a URL
string  Uri.UnescapeDataString(string s)         // reverse
string  WebUtility.HtmlEncode(string s)          // escape for HTML text
string  WebUtility.HtmlDecode(string s)          // reverse

Description#

Base64 — Convert.ToBase64String turns a Binary value (bytes, e.g. a file read via <span class="planned" title="this page is planned and not written yet">storage-file</span> or a Binary property) into standard base64 text; Convert.FromBase64String reverses it. Use it to carry bytes inside a JSON body or a text field.

URL escaping — Uri.EscapeDataString percent-encodes a string so it is safe inside a URL query value or path segment (RFC 3986: a space becomes %20, & becomes %26, and so on). Uri.UnescapeDataString reverses it. Pair it with Http.* when building a request URL from user input:

var url = "https://api.example.com/search?q=" + Uri.EscapeDataString(term);

HTML escaping — WebUtility.HtmlEncode escapes &, <, >, " so a string is safe to place in HTML text without injecting markup; WebUtility.HtmlDecode reverses it.

Hex — Convert.ToHexString answers UPPER-case hex, as .NET does (Crypto.ToHex answers lower case, and Convert.ToHexStringLower is .NET 9's spelling of that); Convert.FromHexString takes either case and throws on text that is not hex.

Text encodings — Encoding.UTF8, ASCII, Latin1, Unicode (UTF-16), BigEndianUnicode, UTF32, Default (UTF-8, as on .NET), and any legacy CODE PAGE through Encoding.GetEncoding(<number or name>): GetBytes, GetString (also over a window, GetString(bytes, index, count)), GetByteCount, GetCharCount, GetPreamble, and the WebName, EncodingName, CodePage and IsSingleByte properties, as in C#. A file format that fixes its encoding is the usual reason — a Swedish SIE bookkeeping file is code page 437 (PC8), a Windows export 1252.

An encoding is an ordinary value: Encoding enc = Encoding.UTF8;, a class field (private readonly Encoding _pc8 = Encoding.GetEncoding(437);), a parameter. Two compare as C#'s cached encodings do — Encoding.GetEncoding(65001) == Encoding.UTF8 is true.

C#'s answers, fallbacks included. A character the encoding cannot hold becomes ? — one per UTF-16 unit, so Encoding.ASCII.GetBytes("a😀") is 61 3F 3F; Latin-1 writes its best-fit letter where .NET has one (Ā → A) — and bytes that are not valid UTF-8 decode to U+FFFD, exactly as in C#. A format where a lost character is a wrong file checks the round trip, as it would in C#: enc.GetString(enc.GetBytes(s)) == s. A name .NET does not know throws ArgumentException.

UTF-8, ASCII, Latin-1, UTF-16 and UTF-32 run in the browser too; a legacy code page is the server's table, and the browser refuses it by name (NotSupportedException), so write a code-page conversion in a server function.

Examples#

Build a signed-looking token payload as base64:

string Encode(byte[] payload) {
  return Convert.ToBase64String(payload);
}

byte[] Decode(string base64) {
  return Convert.FromBase64String(base64);
}

Escape user input into an outbound request (Http.*):

app Shop {
  model "model/**/*.osy";
  use Osysharp.Http;         // `use` is a MANIFEST declaration — it belongs in app.osy
}

string Search(string term) {
  var url = "https://api.example.com/search?q=" + Uri.EscapeDataString(term);
  var r = Http.Get(url);
  return r.Body;
}

Write text in the code page a file format requires, and read it back:

/// An SIE file is PC8 — code page 437 — so "Kassa å" must be written with 437's bytes.
byte[] Pc8(string text) { return Encoding.GetEncoding(437).GetBytes(text); }

string FromPc8(byte[] bytes) { return Encoding.GetEncoding(437).GetString(bytes); }

/// "Kassa å" in PC8 is 4B 61 73 73 61 20 86 — `å` is one byte, 0x86.
string Pc8Hex(string text) { return Convert.ToHexString(Pc8(text)); }

int Utf8Length(string text) { return Encoding.UTF8.GetByteCount(text); }

Hold an encoding in a field and use it from a method — the C# shape of a reader for one file format:

class SieWriter {
  private readonly Encoding _pc8 = Encoding.GetEncoding(437);
  public byte[] Line(string text) => _pc8.GetBytes(text + "\r\n");
}

/// "4B617373612086" then the line end "0D0A" — `å` is ONE byte in PC8, 0x86.
string SieLineHex(string text) => Convert.ToHexString(new SieWriter().Line(text));

/// True — GetEncoding(65001) is UTF-8, the same encoding as the property.
bool SameEncoding() => Encoding.GetEncoding(65001) == Encoding.UTF8;

/// "613F3F": ASCII writes one "?" for each UTF-16 unit it cannot hold, and an emoji is two.
string AsciiOfEmoji() => Convert.ToHexString(Encoding.ASCII.GetBytes("a😀"));

Render user text safely into an HTML fragment:

string Cell(string userText) {
  return "<td>" + WebUtility.HtmlEncode(userText) + "</td>";
}

See also#

• Uri — parse a URL into its parts (new Uri(url).Host)
• Regex — pattern matching, replacing, and splitting text
• Http.* — the outbound HTTP surface these escapings feed