Strings and Unicode: rune, byte and the strings Package

Learning Objectives

Core Concepts

1.1 What string Is Under the Hood and Why It Is Immutable

In Go, a string is an immutable sequence of bytes. Under the hood, a string is effectively a read-only byte slice ([]byte), holding a pointer to the underlying byte array and a length.

Why is it designed this way? 1. Safety: immutability means strings can be shared safely across goroutines without locks. 2. Performance: when a string is used as a map key, hashing can be optimized; and string slicing (e.g. s[1:3]) usually just creates a new header sharing the underlying memory, at almost no cost. 3. Consistency: it fits Go's "explicit over implicit" philosophy and prevents accidental modification.

Note: len(s) returns the byte length, not the character count. For strings containing non-ASCII characters (such as Chinese characters), the byte length is usually larger than the character count.

1.2 rune vs byte, and for range Decoding

Go offers two types for dealing with characters: - byte: an alias of uint8, representing a single byte. - rune: an alias of int32, representing a Unicode code point.

The UTF-8 encoding of one Unicode code point occupies 1–4 bytes; a rune itself is int32, always 4 bytes in memory.

The decoding mechanism of for range When you iterate over a string with for range, the Go runtime automatically performs UTF-8 decoding:

for i, r := range s {
    // i: the byte index of the current rune in the string
    // r: the decoded rune value
}

If the string contains an invalid UTF-8 sequence, r becomes a special replacement character and i advances by 1 byte to skip the erroneous part.

1.3 The strings Package and strings.Builder

The strings package provides a large number of efficient string-processing functions. For cases where strings are concatenated frequently, never use the + operator (which produces a flood of temporary strings); instead use strings.Builder.

The String() method of strings.Builder requires no extra copy: it converts the internal byte slice into a string zero-copy through unsafe (which is why a Builder must not be copied — the runtime detects this and panics). This is the core advantage of strings.Builder over bytes.Buffer.

Example Code

Example 1: Iterating a String by Bytes and by Characters

This example shows how to iterate over a string, distinguishing byte indices, byte values and decoded rune values.

package main

import (
    "fmt"
)

func main() {
    // UTF-8 encoding of "Go 中":
    // 'G'  : 0x47
    // 'o'  : 0x6F
    // ' '  : 0x20
    // '中' : 0xE4 0xB8 0xAD (3 bytes)
    s := "Go 中"

    fmt.Println("Byte iteration:")
    for i := 0; i < len(s); i++ {
        fmt.Printf("Index %d: byte %d (0x%02x)\n", i, s[i], s[i])
    }

    fmt.Println("\nRune iteration:")
    for i, r := range s {
        fmt.Printf("Index %d: rune %c (U+%04X)\n", i, r, r)
    }

    fmt.Printf("\nByte length: %d\n", len(s))
    fmt.Printf("Rune count: %d\n", len([]rune(s)))
}

Expected output:

Byte iteration:
Index 0: byte 71 (0x47)
Index 1: byte 111 (0x6f)
Index 2: byte 32 (0x20)
Index 3: byte 228 (0xe4)
Index 4: byte 184 (0xb8)
Index 5: byte 173 (0xad)

Rune iteration:
Index 0: rune G (U+0047)
Index 1: rune o (U+006F)
Index 2: rune   (U+0020)
Index 3: rune 中 (U+4E2D)

Byte length: 6
Rune count: 4

Example 2: Efficient Concatenation with strings.Builder

This example shows how to use strings.Builder to build a long string, and demonstrates its zero-copy property and common APIs.

package main

import (
    "fmt"
    "strings"
)

func main() {
    var sb strings.Builder

    // Preallocate memory to reduce growths
    sb.Grow(81) // 9*8 + 9 = 81

    for i := 1; i <= 10; i++ {
        sb.WriteString(fmt.Sprintf("Item %d, ", i))
    }

    // String() returns the result zero-copy
    result := sb.String()

    fmt.Printf("Result length: %d\n", len(result))
    fmt.Printf("First 20 chars: %s\n", result[:20])

    // Demonstrate common strings package APIs
    s := "Hello, Go! Hello, World!"

    // Find a substring
    idx := strings.Index(s, "Go")
    fmt.Printf("Index of 'Go': %d\n", idx)

    // Split
    parts := strings.Split(s, ", ")
    fmt.Printf("Split: %v\n", parts)

    // Replace
    replaced := strings.ReplaceAll(s, "Hello", "Hi")
    fmt.Printf("Replaced: %s\n", replaced)
}

Expected output:

Result length: 81
First 20 chars: Item 1, Item 2, Item 3
Index of 'Go': 7
Split: [Hello, Go! Hello, World!]
Replaced: Hi, Go! Hi, World!

Common Mistakes & Pitfalls

Best Practices

Exercises

1. Concept question Explain why Go's string type is designed to be immutable. What impact does this have on concurrent programming?

2. Coding exercise Write a function CountRunes(s string) int that counts the number of Unicode characters (i.e. runes) in the string s. You must not use the direct conversion len([]rune(s)); implement it by iterating.

3. Advanced/debugging exercise The code below tries to convert lowercase letters in a string to uppercase, but has a compile or logic error. Find the problem and fix it. ```go package main

   import ( "fmt" "strings" )

   func main() { s := "hello 世界" for i := 0; i < len(s); i++ { // Try to uppercase the current byte s[i] = strings.ToUpper(string(s[i]))[0] } fmt.Println(s) } ```

Reference Answers

package main

import (
    "fmt"
    "unicode/utf8"
)

// CountRunes counts the number of runes in a string
func CountRunes(s string) int {
    count := 0
    for len(s) > 0 {
        _, size := utf8.DecodeRuneInString(s)
        if size == 0 {
            // Handle an invalid UTF-8 sequence by skipping 1 byte
            size = 1
        }
        count++
        s = s[size:]
    }
    return count
}

func main() {
    fmt.Println(CountRunes("Hello 世界")) // prints: 8
}

package main

import (
    "fmt"
)

func main() {
    s := "hello 世界"
    // The correct approach: use strings.ToUpper on the whole string,
    // or convert to []rune and process each rune.
    r := []rune(s)
    for i, char := range r {
        if char >= 'a' && char <= 'z' {
            r[i] = char - 32 // simple uppercasing, ASCII only
        }
    }
    fmt.Println(string(r)) // prints: HELLO 世界
}

Lesson Summary