diff options
| author | alandonovan <adonovan@google.com> | 2021-02-12 16:57:32 -0500 |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-02-12 16:57:32 -0500 |
| commit | ebe61bd709bf23d7baddbb34e79084d7d156be04 (patch) | |
| tree | 2efd7da6ec6557b9aaed7932d19e07e1beade288 /syntax/syntax.go | |
| parent | 0a10e4fe7402e37a43d9b62c15bfeac1cd4ef272 (diff) | |
| download | starlark-go-ebe61bd709bf23d7baddbb34e79084d7d156be04.tar.gz | |
starlark: add 'bytes' data type, for binary strings (#330)
THIS IS AN INCOMPATIBLE LANGUAGE CHANGE; see below
This change defines a 'bytes' data type, an immutable string of
bytes. In this Go implementation of Starlark, ordinary strings
are also strings of bytes, so the behavior of the two is very similar.
However, that is not required by the spec. Other implementations of
Starlark, notably in Java, may use strings of UTF-16 codes for the
ordinary string type, and thus need a distinct type for byte strings.
See testdata/bytes.star for a tour of the API, and some remaining
questions. See the attached issue for an outline of the proposed
spec change. A Java implementation is underway, but is greatly
complicated by Bazel's unfortunate misdecoding of UTF-8 files as
Latin1.
The string.elems iterable view is now indexable.
The old syntax.quote function (which was in fact not used
except in tests) has been replaced by syntax.Quote,
which is similar to Go's strconv.Quote.
This change removes go.starlark.net.lib.proto.Bytes.
IMPORTANT: string literals that previously used hex escapes
\xXX or octal escapes \OOO to denote byte values greater than 127
will now result in a compile error advising you to use \u
escapes instead if you want the UTF-8 encoding of a code point
in the range U+80 to U+FF. A string literal can no longer
denote invalid text, such as the 1-element string formerly
written "\xff".
Updates https://github.com/bazelbuild/starlark/issues/112
Fixes https://github.com/google/starlark-go/issues/222
Diffstat (limited to 'syntax/syntax.go')
| -rw-r--r-- | syntax/syntax.go | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/syntax/syntax.go b/syntax/syntax.go index 8bbf5c0..20b28bb 100644 --- a/syntax/syntax.go +++ b/syntax/syntax.go @@ -251,7 +251,7 @@ func (x *Ident) Span() (start, end Position) { // A Literal represents a literal string or number. type Literal struct { commentsRef - Token Token // = STRING | INT | FLOAT + Token Token // = STRING | BYTES | INT | FLOAT TokenPos Position Raw string // uninterpreted text Value interface{} // = string | int64 | *big.Int | float64 |