aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorAndroid Chromium Automerger <chromium-automerger@android>2014-01-09 12:40:57 +0000
committerAndroid Chromium Automerger <chromium-automerger@android>2014-01-09 12:40:57 +0000
commit8cba613a0b71e71abcab87aa2478cdddf5bef1ef (patch)
tree10b9caaf1a19f4793e3a4e48900c053783e844e8
parent205450ef65e8d014451503882440bb25a0c9f38b (diff)
parentdfc5a9f2151a7c88914c236c7db8fa119fee249c (diff)
downloadsrc-8cba613a0b71e71abcab87aa2478cdddf5bef1ef.tar.gz
Merge third_party/brotli/src from https://chromium.googlesource.com/external/font-compression-reference.git at dfc5a9f2151a7c88914c236c7db8fa119fee249c
This commit was generated by merge_from_chromium.py. Change-Id: Ia04f2b3241d465f54b5f57c92c478560f8aa3e51
-rw-r--r--brotli/brotlispec.txt156
-rw-r--r--brotli/dec/bit_reader.c38
-rw-r--r--brotli/dec/bit_reader.h119
-rw-r--r--brotli/dec/context.h249
-rw-r--r--brotli/dec/decode.c447
-rw-r--r--brotli/dec/decode.h53
-rw-r--r--brotli/dec/huffman.c99
-rw-r--r--brotli/dec/huffman.h63
-rw-r--r--brotli/dec/prefix.h39
-rw-r--r--brotli/dec/safe_malloc.c35
-rw-r--r--brotli/dec/safe_malloc.h50
-rw-r--r--brotli/dec/streams.c37
-rw-r--r--brotli/dec/streams.h69
-rw-r--r--brotli/dec/types.h35
-rw-r--r--brotli/enc/backward_references.cc2
-rw-r--r--brotli/enc/bit_cost.h10
-rw-r--r--brotli/enc/block_splitter.cc40
-rw-r--r--brotli/enc/encode.cc72
-rw-r--r--brotli/enc/entropy_encode.cc6
-rw-r--r--brotli/enc/hash.h8
-rw-r--r--brotli/enc/literal_cost.cc2
21 files changed, 835 insertions, 794 deletions
diff --git a/brotli/brotlispec.txt b/brotli/brotlispec.txt
index 5a32a2d..23de497 100644
--- a/brotli/brotlispec.txt
+++ b/brotli/brotlispec.txt
@@ -48,7 +48,7 @@ Abstract
* Can be produced or consumed, even for an arbitrarily long
sequentially presented input data stream, using only an a
priori bounded amount of intermediate storage, and hence
- can be used in data communications or similar structures
+ can be used in data communications or similar structures,
such as Unix filters;
* Compresses data with efficiency comparable to the best
currently available general-purpose compression methods,
@@ -62,9 +62,8 @@ Abstract
1.2. Intended audience
- This specification is intended for use by implementors of software
- to compress data into "brotli" format and/or decompress data from
- "brotli" format.
+ This specification is intended for use by software implementers
+ to compress data into and/or decompress data from "brotli" format.
The text of the specification assumes a basic background in
programming at the level of bits and other primitive data
@@ -88,7 +87,7 @@ Abstract
Unless otherwise indicated below, a compliant decompressor must be
able to accept and decompress any data set that conforms to all
- the specifications presented here; a compliant compressor must
+ the specifications presented here. A compliant compressor must
produce data sets that conform to all the specifications presented
here.
@@ -97,7 +96,7 @@ Abstract
Byte: 8 bits stored or transmitted as a unit (same as an octet).
For this specification, a byte is exactly 8 bits, even on machines
which store a character on a number of bits different from eight.
- See below, for the numbering of bits within a byte.
+ See below for the numbering of bits within a byte.
String: a sequence of arbitrary bytes.
@@ -135,7 +134,7 @@ Abstract
bits of a byte are transmitted on a bit-sequential medium,
since the final data format described here is byte- rather than
bit-oriented. However, we describe the compressed block format
- in below, as a sequence of data elements of various bit
+ below as a sequence of data elements of various bit
lengths, not a sequence of bytes. We must therefore specify
how to pack these data elements into bytes to form the final
compressed byte sequence:
@@ -161,7 +160,7 @@ Abstract
2. Compressed representation overview
A compressed data set consists of a header and a series of meta-
- blocks, corresponding to successive meta-blocks of input data. The
+ blocks corresponding to successive meta-blocks of input data. The
meta-block sizes are limited to bytes and the maximum meta-block size
is 268,435,456 bytes.
@@ -209,7 +208,7 @@ Abstract
(IaC0, L0, L1, L2, D0)(IaC1, D1)(IaC2, L3, L4, D2)(IaC3, L5, D3)
The meta-block here has 4 commands, and each three types of symbols
- within these commands can be rearranged into for example the
+ within these commands can be rearranged for example into the
following logical block structure:
[IaC0, IaC1][IaC2, IaC3] <-- block types 0 and 1
@@ -243,16 +242,16 @@ Abstract
Each type of value (insert-and-copy lengths, literals and distances)
can be encoded with any Huffman tree from a collection of Huffman
trees of the same kind appearing in the meta-block header. The
- particual Huffman tree used can depend on two factors: the block type
+ particular Huffman tree used can depend on two factors: the block type
of the block the value appears in, and the context of the value. In
the case of the literals, the context is the previous two bytes in
the input data, and in the case of distances, the context is the copy
length from the same command. For insert-and-copy lengths, no context
is used and the Huffman tree depends only on the block type (in fact,
the index of the Huffman tree is the block type number). In the case
- of literals and distances, the context is mapped to a context id in
+ of literals and distances, the context is mapped to a context ID in
the rage [0, 63] for literals and [0, 3] for distances and the matrix
- of the Huffman tree indices for each block type and context id,
+ of the Huffman tree indices for each block type and context ID,
called the context map, is encoded in a compact form in the meta-
block header.
@@ -405,24 +404,24 @@ Abstract
Huffman codes are used for different purposes in the "brotli"
format, and each purpose has a different alphabet size. For
- literal codes, the alphabet size is 256. For insert-and-copy
- length codes, the alphabet size is 704. For block length codes,
+ literal codes the alphabet size is 256. For insert-and-copy
+ length codes the alphabet size is 704. For block length codes,
the alphabet size is 26. For distance codes, block type codes and
the Huffman codes used in compressing the context map, the
alphabet size is dynamic and is based on other parameters.
3.4. Simple Huffman codes
- The first bit of the compressed representation of each Huffman
+ The first two bits of the compressed representation of each Huffman
code distinguishes between simple and complex Huffman codes. If
- the first bit is 1, then a simple, otherwise a complex Huffman
- code follows.
+ this value is 1, then a simple Huffman code follows. Otherwise
+ the value indicates the number of leading zeros.
A simple Huffman code can have only up to four symbols with non-
zero code length. The format of the simple Huffman code is as
follows:
- 1 bit: 1, indicating a simple Huffman code
+ 2 bits: value of 1 indicates a simple Huffman code
2 bits: NSYM - 1, where NSYM = # of symbols with non-zero
code length
@@ -433,7 +432,9 @@ Abstract
The value of ALPHABET_BITS depends on the alphabet of the Huffman
code: it is the smallest number of bits that can represent all
symbols in the alphabet. E.g. for the alphabet of literal bytes,
- ALPHABET_BITS is 8.
+ ALPHABET_BITS is 8. The value of each of the NSYM symbols above is
+ the value of the ALPHABETS_BITS width machine integer representing
+ the symbol modulo the alphabet size of the Huffman code.
The (non-zero) code lengths of the symbols can be reconstructed as
follows:
@@ -488,7 +489,8 @@ Abstract
A code length of 0 indicates that the corresponding symbol in the
alphabet will not occur in the compressed data, and should not
participate in the Huffman code construction algorithm given
- earlier.
+ earlier. A complex Huffman code must have at least two non-zero
+ code lengths.
The bit lengths of the Huffman code over the code length alphabet
are compressed with the following static Huffman code:
@@ -505,53 +507,35 @@ Abstract
We can now define the format of the complex Huffman code as
follows:
- 1 bit: 0, indicating a complex Huffman code
- 4 bits: HCLEN, # of code length codes - 3
- 1 bit : HSKIP, if 1, skip over first two code length codes
+ 2 bits: HSKIP, values of 0, 2 or 3 represent the respective
+ number of leading zeros. (Value of 1 indicates the
+ Simple Huffman code.)
- (HCLEN + 3 - 2 * HSKIP) code lengths for symbols in the code
- length alphabet given just above, in the order: 1, 2, 3,
- 4, 0, 17, 5, 6, 16, 7, 8, 9, 10, 11, 12, 13, 14, 15
- If HSKIP is 1, code lengths of code length symbols 1 and
- 2 are implicit zeros. Code lengths of code length symbols
- beyond the (HCLEN + 4)th in the ordering above are also
- implicit zeros.
+ Code lengths for symbols in the code length alphabet given
+ just above, in the order: 1, 2, 3, 4, 0, 17, 5, 6, 16, 7,
+ 8, 9, 10, 11, 12, 13, 14, 15
The code lengths of code length symbols are between 0 and
5 and they are represented with 2 - 5 bits according to
the static Huffman code above. A code length of 0 means
the corresponding code length symbol is not used.
- 1 bit: HLENINC, if 1, the number of code length symbols is
- encoded next
-
- 7-8 bits: HLEN, # of code length symbols, with the following
- encoding: values 4 - 67 with bit pattern 0xxxxxx,
- values 68 - 195 with bit pattern 1xxxxxxx, appears
- only if HLENINC = 1
+ If HSKIP is 2 or 3, a respective number of leading code
+ lengths are implicit zeros and are not present in the
+ code lengths sequence above. If there are at least two non-
+ zero code lengths, any trailing zero code lengths are
+ omitted, i.e. the last code length in the sequence must
+ be non-zero. In this case the sum of (32 >> code length)
+ over all the non-zero code lengths must equal to 32.
Sequence of code lengths symbols, encoded using the code
- length Huffman code. The number of code length symbols
- is either HLEN (in case of HLENINC = 1), or as many as is
- needed to assign a code length to each symbol in the
- alphabet (i.e. the alphabet size minus the sum of all the
- repeat lengths defined by extra bits of code length
- symbols 16 and 17). In case of HLENINC = 1, all symbols
- not assigned a code length have implicit code length 0.
-
- 3.6. Validity of the Huffman code
-
- There are two kinds of valid Huffman codes:
- * Huffman code that contains one symbol of length 1, and
- * Full canonical Huffman code.
-
- A decoder can check if the Huffman code is full by using integer
- arithmetic, by computing if the sum of (32768 right-shifted by
- code-length) over the non-zero code lengths leads to a total sum
- of 32768. However, if there is only one non-zero code-length, it
- shall have an implicit code length of one and the code is
- considered valid.
+ length Huffman code. Any trailing 0 or 17 must be
+ omitted, i.e. the last encoded code length symbol must be
+ between 1 and 16. The sum of (32768 >> code length) over
+ all the non-zero code lengths in the alphabet, including
+ those encoded using repeat code(s) of 16, must equal to
+ 32768.
4. Encoding of distances
@@ -568,11 +552,11 @@ Abstract
on the distance code.
To convert a distance code and associated extra bits to a backward
- distance, we need the sequence of past distances and two additonal
+ distance, we need the sequence of past distances and two additional
parameters, the number of "postfix bits", denoted by NPOSTFIX, and
the number of direct distance codes, denoted by NDIRECT. Both of
these parameters are encoded in the meta-block header. We will also
- use the folowing derived parameter:
+ use the following derived parameter:
POSTFIX_MASK = ((1 << NPOSTFIX) - 1)
@@ -711,7 +695,7 @@ Abstract
First, look up the cell with the 64 value range containing the
insert-and-copy length code, this gives the insert length code and
- and the copy length code ranges, both 8 values long. The copy length
+ the copy length code ranges, both 8 values long. The copy length
code within its range is determined by the lowest 3 bits of the
insert-and-copy length code, and the insert length code within its
range is determined by bits 3-5 (counted from the LSB) of the insert-
@@ -734,11 +718,13 @@ Abstract
alphabet. A block type code 0 means that the block type is the same
as the type of the second last block from the same block category,
while a block type code 1 means that the block type equals the last
- block type plus one. Block type codes 2 - 257 represent block types
- 0 - 255. The second last and last block types are initialized with 0
- and 1, respectively, at the beginning of each meta-block.
+ block type plus one. If the last block type is the maximal possible,
+ then a block type code 1 means block type 0. Block type codes 2 - 257
+ represent block types 0 - 255. The second last and last block types
+ are initialized with 0 and 1, respectively, at the beginning of each
+ meta-block.
- The first block type of each block category must be 0, and the block
+ The first block type of each block category must be 0 and the block
type of the first block switch command is therefore not encoded in
the compressed data.
@@ -783,13 +769,13 @@ Abstract
7. Context modeling
As described in Section 2, the Huffman tree used to encode a literal
- byte or a distance code depends on the context id and the block type.
- This section specifies how to compute the context id for a particular
+ byte or a distance code depends on the context ID and the block type.
+ This section specifies how to compute the context ID for a particular
literal and distance code, and how to encode the context map that
- maps a <context id, block type> pair to the index of a Huffman
+ maps a <context ID, block type> pair to the index of a Huffman
tree in the array of literal and distance Huffman trees.
- 7.1. Context modes and context id lookup for literals
+ 7.1. Context modes and context ID lookup for literals
The context for encoding the next literal is defined by the last
two bytes in the stream (p1, p2, where p1 is the most recent
@@ -864,8 +850,8 @@ Abstract
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7
- Given p1 = last decoded byte, and p2 = second last decoded byte,
- the context ids can be computed as follows:
+ Given p1 is the last decoded byte and p2 is the second last
+ decoded byte the context IDs can be computed as follows:
For LSB6 : Context ID = p1 & 0x3f
For MSB6 : Context ID = p1 >> 2
@@ -875,14 +861,14 @@ Abstract
The context modes LSB6, MSB6, UTF8, and Signed are denoted by
integers 0, 1, 2, 3.
- The context mode is defined for each literal block type, and they
- are stored in a consequtive array of bits in the meta-block
+ The context mode is defined for each literal block type and they
+ are stored in a consecutive array of bits in the meta-block
header, always two bits per block type.
- 7.2. Context id for distances
+ 7.2. Context ID for distances
The context for encoding a distance code is defined by the copy
- length corresponding to the distance. The context ids are 0, 1, 2,
+ length corresponding to the distance. The context IDs are 0, 1, 2,
and 3 for copy lengths 2, 3, 4, and more than 4, respectively.
7.3. Encoding of the context map
@@ -898,7 +884,7 @@ Abstract
CMAPL[0..(64 * NBLTYPESL - 1)] and CMAPD[0..(4 * NBLTYPESD - 1)].
The index of the Huffman tree for encoding a literal or distance
- code with context id "cid" and block type "bltype" is
+ code with context ID "cid" and block type "bltype" is
index of literal Huffman tree = CMAPL[bltype * 64 + cid]
@@ -946,7 +932,7 @@ Abstract
At any given point during decoding the compressed data, a reference
to a duplicated string in the output produced so far has a maximum
- backward distance value, which is the minumum of the window size and
+ backward distance value, which is the minimum of the window size and
the number of output bytes produced. However, decoding a distance
from the input stream, as described in section 4, can produce
distances that are greater than this maximum allowed value. The
@@ -994,7 +980,7 @@ Abstract
In this section we describe the format of the compressed data set in
terms of the format of the individual data items described in the
- previous secions.
+ previous sections.
9.1. Format of the stream header
@@ -1013,7 +999,7 @@ Abstract
9.2. Format of the meta-block header
A compliant compressed data set has at least one meta-block. Each
- meta-block contains a header, with information about the
+ meta-block contains a header with information about the
uncompressed length of the meta-block, and a bit signaling if the
meta-block is the last one. The format of the meta-block header is
the following:
@@ -1214,7 +1200,7 @@ Abstract
read block length using HTREE_BLEN_L and set BLEN_L
decrement BLEN_L
look up context mode CMODE[BTYPE_L]
- compute context id, CIDL from last two bytes of output
+ compute context ID, CIDL from last two bytes of output
read literal using HTREEL[CMAPL[64 * BTYPE_L + CIDL]]
copy literal to output stream
if number of output bytes produced in the loop is MLEN
@@ -1226,7 +1212,7 @@ Abstract
read block type using HTREE_BTYPE_D and set BTYPE_D
read block length using HTREE_BLEN_D and set BLEN_D
decrement BLEN_D
- compute context id, CIDD from CLEN
+ compute context ID, CIDD from CLEN
read distance code with HTREED[CMAPD[4 * BTYPE_D + CIDD]]
compute distance by distance short code substitution
move backwards distance bytes in the output stream, and
@@ -1237,12 +1223,12 @@ Abstract
while not ISLAST
Note that a duplicated string reference may refer to a string in a
- previous meta-block; i.e., the backward distance may cross one or
+ previous meta-block, i.e. the backward distance may cross one or
more meta-block boundaries. However a backward copy distance
- cannot refer past the beginning of the output stream, and it can
- not be greater than the window size, any such distance must be
- interpreted as a reference to a static dictionary word. Note also
- that the referenced string may overlap the current position; for
+ cannot refer past the beginning of the output stream and it can
+ not be greater than the window size; any such distance must be
+ interpreted as a reference to a static dictionary word. Also note
+ that the referenced string may overlap the current position, for
example, if the last 2 bytes decoded have values X and Y, a string
reference with <length = 5, distance = 2> adds X,Y,X,Y,X to the
output stream.
diff --git a/brotli/dec/bit_reader.c b/brotli/dec/bit_reader.c
index 9781248..25e33e3 100644
--- a/brotli/dec/bit_reader.c
+++ b/brotli/dec/bit_reader.c
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Bit reading helpers
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Bit reading helpers
+*/
#include <assert.h>
#include <stdlib.h>
@@ -27,11 +28,12 @@ int BrotliInitBitReader(BrotliBitReader* const br, BrotliInput input) {
size_t i;
assert(br != NULL);
+ br->buf_ptr_ = br->buf_;
br->input_ = input;
br->val_ = 0;
br->pos_ = 0;
br->bit_pos_ = 0;
- br->end_pos_ = 0;
+ br->bits_left_ = 64;
br->eos_ = 0;
if (!BrotliReadMoreInput(br)) {
return 0;
@@ -40,9 +42,9 @@ int BrotliInitBitReader(BrotliBitReader* const br, BrotliInput input) {
br->val_ |= ((uint64_t)br->buf_[br->pos_]) << (8 * i);
++br->pos_;
}
- return (br->end_pos_ > 0);
+ return (br->bits_left_ > 64);
}
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
diff --git a/brotli/dec/bit_reader.h b/brotli/dec/bit_reader.h
index 96be036..551cc14 100644
--- a/brotli/dec/bit_reader.h
+++ b/brotli/dec/bit_reader.h
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Bit reading helpers
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Bit reading helpers
+*/
#ifndef BROTLI_DEC_BIT_READER_H_
#define BROTLI_DEC_BIT_READER_H_
@@ -38,29 +39,31 @@ static const uint32_t kBitMask[BROTLI_MAX_NUM_BIT_READ] = {
};
typedef struct {
- // Input byte buffer, consist of a ringbuffer and a "slack" region where
- // bytes from the start of the ringbuffer are copied.
+ /* Input byte buffer, consist of a ringbuffer and a "slack" region where */
+ /* bytes from the start of the ringbuffer are copied. */
uint8_t buf_[BROTLI_IBUF_SIZE];
- BrotliInput input_; // input callback
- uint64_t val_; // pre-fetched bits
- size_t pos_; // byte position in stream
- int bit_pos_; // current bit-reading position in val_
- size_t end_pos_; // current end position in stream
- int eos_; // input stream is finished
+ uint8_t* buf_ptr_; /* next input will write here */
+ BrotliInput input_; /* input callback */
+ uint64_t val_; /* pre-fetched bits */
+ uint32_t pos_; /* byte position in stream */
+ uint32_t bit_pos_; /* current bit-reading position in val_ */
+ uint32_t bits_left_; /* how many valid bits left */
+ int eos_; /* input stream is finished */
} BrotliBitReader;
int BrotliInitBitReader(BrotliBitReader* const br, BrotliInput input);
-// Return the prefetched bits, so they can be looked up.
+/* Return the prefetched bits, so they can be looked up. */
static BROTLI_INLINE uint32_t BrotliPrefetchBits(BrotliBitReader* const br) {
return (uint32_t)(br->val_ >> br->bit_pos_);
}
-// For jumping over a number of bits in the bit stream when accessed with
-// BrotliPrefetchBits and BrotliFillBitWindow.
-static BROTLI_INLINE void BrotliSetBitPos(BrotliBitReader* const br, int val) {
+/* For jumping over a number of bits in the bit stream when accessed with */
+/* BrotliPrefetchBits and BrotliFillBitWindow. */
+static BROTLI_INLINE void BrotliSetBitPos(BrotliBitReader* const br,
+ uint32_t val) {
#ifdef BROTLI_DECODE_DEBUG
- int n_bits = val - br->bit_pos_;
+ uint32_t n_bits = val - br->bit_pos_;
const uint32_t bval = (uint32_t)(br->val_ >> br->bit_pos_) & kBitMask[n_bits];
printf("[BrotliReadBits] %010ld %2d val: %6x\n",
(br->pos_ << 3) + br->bit_pos_ - 64, n_bits, bval);
@@ -68,41 +71,43 @@ static BROTLI_INLINE void BrotliSetBitPos(BrotliBitReader* const br, int val) {
br->bit_pos_ = val;
}
-// Reload up to 64 bits byte-by-byte
+/* Reload up to 64 bits byte-by-byte */
static BROTLI_INLINE void ShiftBytes(BrotliBitReader* const br) {
while (br->bit_pos_ >= 8) {
br->val_ >>= 8;
br->val_ |= ((uint64_t)br->buf_[br->pos_ & BROTLI_IBUF_MASK]) << 56;
++br->pos_;
br->bit_pos_ -= 8;
+ br->bits_left_ -= 8;
}
}
-// Fills up the input ringbuffer by calling the input callback.
-//
-// Does nothing if there are at least 32 bytes present after current position.
-//
-// Returns 0 if either:
-// - the input callback returned an error, or
-// - there is no more input and the position is past the end of the stream.
-//
-// After encountering the end of the input stream, 32 additional zero bytes are
-// copied to the ringbuffer, therefore it is safe to call this function after
-// every 32 bytes of input is read.
+/* Fills up the input ringbuffer by calling the input callback.
+
+ Does nothing if there are at least 32 bytes present after current position.
+
+ Returns 0 if either:
+ - the input callback returned an error, or
+ - there is no more input and the position is past the end of the stream.
+
+ After encountering the end of the input stream, 32 additional zero bytes are
+ copied to the ringbuffer, therefore it is safe to call this function after
+ every 32 bytes of input is read.
+*/
static BROTLI_INLINE int BrotliReadMoreInput(BrotliBitReader* const br) {
- if (br->pos_ + 32 < br->end_pos_) {
+ if (br->bits_left_ > 320) {
return 1;
} else if (br->eos_) {
- return (br->pos_ << 3) + br->bit_pos_ <= (br->end_pos_ << 3) + 64;
+ return br->bit_pos_ <= br->bits_left_;
} else {
- uint8_t* dst = br->buf_ + (br->end_pos_ & BROTLI_IBUF_MASK);
+ uint8_t* dst = br->buf_ptr_;
int bytes_read = BrotliRead(br->input_, dst, BROTLI_READ_SIZE);
if (bytes_read < 0) {
return 0;
}
if (bytes_read < BROTLI_READ_SIZE) {
br->eos_ = 1;
- // Store 32 bytes of zero after the stream end.
+ /* Store 32 bytes of zero after the stream end. */
#if (defined(__x86_64__) || defined(_M_X64))
*(uint64_t*)(dst + bytes_read) = 0;
*(uint64_t*)(dst + bytes_read + 8) = 0;
@@ -113,7 +118,7 @@ static BROTLI_INLINE int BrotliReadMoreInput(BrotliBitReader* const br) {
#endif
}
if (dst == br->buf_) {
- // Copy the head of the ringbuffer to the slack region.
+ /* Copy the head of the ringbuffer to the slack region. */
#if (defined(__x86_64__) || defined(_M_X64))
UNALIGNED_COPY64(br->buf_ + BROTLI_IBUF_SIZE - 32, br->buf_);
UNALIGNED_COPY64(br->buf_ + BROTLI_IBUF_SIZE - 24, br->buf_ + 8);
@@ -122,31 +127,35 @@ static BROTLI_INLINE int BrotliReadMoreInput(BrotliBitReader* const br) {
#else
memcpy(br->buf_ + (BROTLI_READ_SIZE << 1), br->buf_, 32);
#endif
+ br->buf_ptr_ = br->buf_ + BROTLI_READ_SIZE;
+ } else {
+ br->buf_ptr_ = br->buf_;
}
- br->end_pos_ += bytes_read;
+ br->bits_left_ += ((uint32_t)bytes_read << 3);
return 1;
}
}
-// Advances the Read buffer by 5 bytes to make room for reading next 24 bits.
+/* Advances the Read buffer by 5 bytes to make room for reading next 24 bits. */
static BROTLI_INLINE void BrotliFillBitWindow(BrotliBitReader* const br) {
if (br->bit_pos_ >= 40) {
#if (defined(__x86_64__) || defined(_M_X64))
br->val_ >>= 40;
- br->bit_pos_ -= 40;
- // The expression below needs a little-endian arch to work correctly.
- // This gives a large speedup for decoding speed.
+ /* The expression below needs a little-endian arch to work correctly. */
+ /* This gives a large speedup for decoding speed. */
br->val_ |= *(const uint64_t*)(
br->buf_ + (br->pos_ & BROTLI_IBUF_MASK)) << 24;
br->pos_ += 5;
+ br->bit_pos_ -= 40;
+ br->bits_left_ -= 40;
#else
ShiftBytes(br);
#endif
}
}
-// Reads the specified number of bits from Read Buffer.
-// Requires that n_bits is positive.
+/* Reads the specified number of bits from Read Buffer. */
+/* Requires that n_bits is positive. */
static BROTLI_INLINE uint32_t BrotliReadBits(
BrotliBitReader* const br, int n_bits) {
uint32_t val;
@@ -156,12 +165,12 @@ static BROTLI_INLINE uint32_t BrotliReadBits(
printf("[BrotliReadBits] %010ld %2d val: %6x\n",
(br->pos_ << 3) + br->bit_pos_ - 64, n_bits, val);
#endif
- br->bit_pos_ += n_bits;
+ br->bit_pos_ += (uint32_t)n_bits;
return val;
}
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
-#endif // BROTLI_DEC_BIT_READER_H_
+#endif /* BROTLI_DEC_BIT_READER_H_ */
diff --git a/brotli/dec/context.h b/brotli/dec/context.h
index 212a445..dbc0c36 100644
--- a/brotli/dec/context.h
+++ b/brotli/dec/context.h
@@ -1,107 +1,108 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Lookup table to map the previous two bytes to a context id.
-//
-// There are four different context modeling modes defined here:
-// CONTEXT_LSB6: context id is the least significant 6 bits of the last byte,
-// CONTEXT_MSB6: context id is the most significant 6 bits of the last byte,
-// CONTEXT_UTF8: second-order context model tuned for UTF8-encoded text,
-// CONTEXT_SIGNED: second-order context model tuned for signed integers.
-//
-// The context id for the UTF8 context model is calculated as follows. If p1
-// and p2 are the previous two bytes, we calcualte the context as
-//
-// context = kContextLookup[p1] | kContextLookup[p2 + 256].
-//
-// If the previous two bytes are ASCII characters (i.e. < 128), this will be
-// equivalent to
-//
-// context = 4 * context1(p1) + context2(p2),
-//
-// where context1 is based on the previous byte in the following way:
-//
-// 0 : non-ASCII control
-// 1 : \t, \n, \r
-// 2 : space
-// 3 : other punctuation
-// 4 : " '
-// 5 : %
-// 6 : ( < [ {
-// 7 : ) > ] }
-// 8 : , ; :
-// 9 : .
-// 10 : =
-// 11 : number
-// 12 : upper-case vowel
-// 13 : upper-case consonant
-// 14 : lower-case vowel
-// 15 : lower-case consonant
-//
-// and context2 is based on the second last byte:
-//
-// 0 : control, space
-// 1 : punctuation
-// 2 : upper-case letter, number
-// 3 : lower-case letter
-//
-// If the last byte is ASCII, and the second last byte is not (in a valid UTF8
-// stream it will be a continuation byte, value between 128 and 191), the
-// context is the same as if the second last byte was an ASCII control or space.
-//
-// If the last byte is a UTF8 lead byte (value >= 192), then the next byte will
-// be a continuation byte and the context id is 2 or 3 depending on the LSB of
-// the last byte and to a lesser extent on the second last byte if it is ASCII.
-//
-// If the last byte is a UTF8 continuation byte, the second last byte can be:
-// - continuation byte: the next byte is probably ASCII or lead byte (assuming
-// 4-byte UTF8 characters are rare) and the context id is 0 or 1.
-// - lead byte (192 - 207): next byte is ASCII or lead byte, context is 0 or 1
-// - lead byte (208 - 255): next byte is continuation byte, context is 2 or 3
-//
-// The possible value combinations of the previous two bytes, the range of
-// context ids and the type of the next byte is summarized in the table below:
-//
-// |--------\-----------------------------------------------------------------|
-// | \ Last byte |
-// | Second \---------------------------------------------------------------|
-// | last byte \ ASCII | cont. byte | lead byte |
-// | \ (0-127) | (128-191) | (192-) |
-// |=============|===================|=====================|==================|
-// | ASCII | next: ASCII/lead | not valid | next: cont. |
-// | (0-127) | context: 4 - 63 | | context: 2 - 3 |
-// |-------------|-------------------|---------------------|------------------|
-// | cont. byte | next: ASCII/lead | next: ASCII/lead | next: cont. |
-// | (128-191) | context: 4 - 63 | context: 0 - 1 | context: 2 - 3 |
-// |-------------|-------------------|---------------------|------------------|
-// | lead byte | not valid | next: ASCII/lead | not valid |
-// | (192-207) | | context: 0 - 1 | |
-// |-------------|-------------------|---------------------|------------------|
-// | lead byte | not valid | next: cont. | not valid |
-// | (208-) | | context: 2 - 3 | |
-// |-------------|-------------------|---------------------|------------------|
-//
-// The context id for the signed context mode is calculated as:
-//
-// context = (kContextLookup[512 + p1] << 3) | kContextLookup[512 + p2].
-//
-// For any context modeling modes, the context ids can be calculated by |-ing
-// together two lookups from one table using context model dependent offsets:
-//
-// context = kContextLookup[offset1 + p1] | kContextLookup[offset2 + p2].
-//
-// where offset1 and offset2 are dependent on the context mode.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Lookup table to map the previous two bytes to a context id.
+
+ There are four different context modeling modes defined here:
+ CONTEXT_LSB6: context id is the least significant 6 bits of the last byte,
+ CONTEXT_MSB6: context id is the most significant 6 bits of the last byte,
+ CONTEXT_UTF8: second-order context model tuned for UTF8-encoded text,
+ CONTEXT_SIGNED: second-order context model tuned for signed integers.
+
+ The context id for the UTF8 context model is calculated as follows. If p1
+ and p2 are the previous two bytes, we calcualte the context as
+
+ context = kContextLookup[p1] | kContextLookup[p2 + 256].
+
+ If the previous two bytes are ASCII characters (i.e. < 128), this will be
+ equivalent to
+
+ context = 4 * context1(p1) + context2(p2),
+
+ where context1 is based on the previous byte in the following way:
+
+ 0 : non-ASCII control
+ 1 : \t, \n, \r
+ 2 : space
+ 3 : other punctuation
+ 4 : " '
+ 5 : %
+ 6 : ( < [ {
+ 7 : ) > ] }
+ 8 : , ; :
+ 9 : .
+ 10 : =
+ 11 : number
+ 12 : upper-case vowel
+ 13 : upper-case consonant
+ 14 : lower-case vowel
+ 15 : lower-case consonant
+
+ and context2 is based on the second last byte:
+
+ 0 : control, space
+ 1 : punctuation
+ 2 : upper-case letter, number
+ 3 : lower-case letter
+
+ If the last byte is ASCII, and the second last byte is not (in a valid UTF8
+ stream it will be a continuation byte, value between 128 and 191), the
+ context is the same as if the second last byte was an ASCII control or space.
+
+ If the last byte is a UTF8 lead byte (value >= 192), then the next byte will
+ be a continuation byte and the context id is 2 or 3 depending on the LSB of
+ the last byte and to a lesser extent on the second last byte if it is ASCII.
+
+ If the last byte is a UTF8 continuation byte, the second last byte can be:
+ - continuation byte: the next byte is probably ASCII or lead byte (assuming
+ 4-byte UTF8 characters are rare) and the context id is 0 or 1.
+ - lead byte (192 - 207): next byte is ASCII or lead byte, context is 0 or 1
+ - lead byte (208 - 255): next byte is continuation byte, context is 2 or 3
+
+ The possible value combinations of the previous two bytes, the range of
+ context ids and the type of the next byte is summarized in the table below:
+
+ |--------\-----------------------------------------------------------------|
+ | \ Last byte |
+ | Second \---------------------------------------------------------------|
+ | last byte \ ASCII | cont. byte | lead byte |
+ | \ (0-127) | (128-191) | (192-) |
+ |=============|===================|=====================|==================|
+ | ASCII | next: ASCII/lead | not valid | next: cont. |
+ | (0-127) | context: 4 - 63 | | context: 2 - 3 |
+ |-------------|-------------------|---------------------|------------------|
+ | cont. byte | next: ASCII/lead | next: ASCII/lead | next: cont. |
+ | (128-191) | context: 4 - 63 | context: 0 - 1 | context: 2 - 3 |
+ |-------------|-------------------|---------------------|------------------|
+ | lead byte | not valid | next: ASCII/lead | not valid |
+ | (192-207) | | context: 0 - 1 | |
+ |-------------|-------------------|---------------------|------------------|
+ | lead byte | not valid | next: cont. | not valid |
+ | (208-) | | context: 2 - 3 | |
+ |-------------|-------------------|---------------------|------------------|
+
+ The context id for the signed context mode is calculated as:
+
+ context = (kContextLookup[512 + p1] << 3) | kContextLookup[512 + p2].
+
+ For any context modeling modes, the context ids can be calculated by |-ing
+ together two lookups from one table using context model dependent offsets:
+
+ context = kContextLookup[offset1 + p1] | kContextLookup[offset2 + p2].
+
+ where offset1 and offset2 are dependent on the context mode.
+*/
#ifndef BROTLI_DEC_CONTEXT_H_
#define BROTLI_DEC_CONTEXT_H_
@@ -115,11 +116,10 @@ enum ContextType {
CONTEXT_SIGNED = 3
};
-// Common context lookup table for all context modes.
+/* Common context lookup table for all context modes. */
static const uint8_t kContextLookup[1792] = {
- // CONTEXT_UTF8, last byte.
- //
- // ASCII range.
+ /* CONTEXT_UTF8, last byte. */
+ /* ASCII range. */
0, 0, 0, 0, 0, 0, 0, 0, 0, 4, 4, 0, 0, 4, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8, 12, 16, 12, 12, 20, 12, 16, 24, 28, 12, 12, 32, 12, 36, 12,
@@ -128,19 +128,18 @@ static const uint8_t kContextLookup[1792] = {
52, 52, 52, 52, 52, 48, 52, 52, 52, 52, 52, 24, 12, 28, 12, 12,
12, 56, 60, 60, 60, 56, 60, 60, 60, 56, 60, 60, 60, 60, 60, 56,
60, 60, 60, 60, 60, 56, 60, 60, 60, 60, 60, 24, 12, 28, 12, 0,
- // UTF8 continuation byte range.
+ /* UTF8 continuation byte range. */
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1, 0, 1,
- // UTF8 lead byte range.
+ /* UTF8 lead byte range. */
2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3, 2, 3,
- // CONTEXT_UTF8 second last byte.
- //
- // ASCII range.
+ /* CONTEXT_UTF8 second last byte. */
+ /* ASCII range. */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
@@ -149,17 +148,17 @@ static const uint8_t kContextLookup[1792] = {
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 1, 1, 1, 1, 1,
1, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 1, 1, 1, 1, 0,
- // UTF8 continuation byte range.
+ /* UTF8 continuation byte range. */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
- // UTF8 lead byte range.
+ /* UTF8 lead byte range. */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
- // CONTEXT_SIGNED, second last byte.
+ /* CONTEXT_SIGNED, second last byte. */
0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
@@ -176,7 +175,7 @@ static const uint8_t kContextLookup[1792] = {
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 6, 7,
- // CONTEXT_SIGNED, last byte, same as the above values shifted by 3 bits.
+ /* CONTEXT_SIGNED, last byte, same as the above values shifted by 3 bits. */
0, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8, 8,
16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16, 16,
@@ -193,7 +192,7 @@ static const uint8_t kContextLookup[1792] = {
40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40, 40,
48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 48, 56,
- // CONTEXT_LSB6, last byte.
+ /* CONTEXT_LSB6, last byte. */
0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15,
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
@@ -210,7 +209,7 @@ static const uint8_t kContextLookup[1792] = {
16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31,
32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47,
48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63,
- // CONTEXT_MSB6, last byte.
+ /* CONTEXT_MSB6, last byte. */
0, 0, 0, 0, 1, 1, 1, 1, 2, 2, 2, 2, 3, 3, 3, 3,
4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 6, 6, 7, 7, 7, 7,
8, 8, 8, 8, 9, 9, 9, 9, 10, 10, 10, 10, 11, 11, 11, 11,
@@ -227,7 +226,7 @@ static const uint8_t kContextLookup[1792] = {
52, 52, 52, 52, 53, 53, 53, 53, 54, 54, 54, 54, 55, 55, 55, 55,
56, 56, 56, 56, 57, 57, 57, 57, 58, 58, 58, 58, 59, 59, 59, 59,
60, 60, 60, 60, 61, 61, 61, 61, 62, 62, 62, 62, 63, 63, 63, 63,
- // CONTEXT_{M,L}SB6, second last byte,
+ /* CONTEXT_{M,L}SB6, second last byte, */
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
@@ -247,14 +246,14 @@ static const uint8_t kContextLookup[1792] = {
};
static const int kContextLookupOffsets[8] = {
- // CONTEXT_LSB6
+ /* CONTEXT_LSB6 */
1024, 1536,
- // CONTEXT_MSB6
+ /* CONTEXT_MSB6 */
1280, 1536,
- // CONTEXT_UTF8
+ /* CONTEXT_UTF8 */
0, 256,
- // CONTEXT_SIGNED
+ /* CONTEXT_SIGNED */
768, 512,
};
-#endif // BROTLI_DEC_CONTEXT_H_
+#endif /* BROTLI_DEC_CONTEXT_H_ */
diff --git a/brotli/dec/decode.c b/brotli/dec/decode.c
index f7ae1df..a8e41ab 100644
--- a/brotli/dec/decode.c
+++ b/brotli/dec/decode.c
@@ -1,16 +1,17 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+*/
#include <stdlib.h>
#include <stdio.h>
@@ -37,8 +38,8 @@ extern "C" {
#define BROTLI_LOG_ARRAY_INDEX(array_name, idx)
#endif
-static const int kDefaultCodeLength = 8;
-static const int kCodeLengthRepeatCode = 16;
+static const uint8_t kDefaultCodeLength = 8;
+static const uint8_t kCodeLengthRepeatCode = 16;
static const int kNumLiteralCodes = 256;
static const int kNumInsertAndCopyCodes = 704;
static const int kNumBlockLengthCodes = 26;
@@ -61,59 +62,59 @@ static const int kDistanceShortCodeValueOffset[NUM_DISTANCE_SHORT_CODES] = {
static BROTLI_INLINE int DecodeWindowBits(BrotliBitReader* br) {
if (BrotliReadBits(br, 1)) {
- return 17 + BrotliReadBits(br, 3);
+ return 17 + (int)BrotliReadBits(br, 3);
} else {
return 16;
}
}
-// Decodes a number in the range [0..255], by reading 1 - 11 bits.
+/* Decodes a number in the range [0..255], by reading 1 - 11 bits. */
static BROTLI_INLINE int DecodeVarLenUint8(BrotliBitReader* br) {
if (BrotliReadBits(br, 1)) {
- int nbits = BrotliReadBits(br, 3);
+ int nbits = (int)BrotliReadBits(br, 3);
if (nbits == 0) {
return 1;
} else {
- return BrotliReadBits(br, nbits) + (1 << nbits);
+ return (int)BrotliReadBits(br, nbits) + (1 << nbits);
}
}
return 0;
}
static void DecodeMetaBlockLength(BrotliBitReader* br,
- size_t* meta_block_length,
+ int* meta_block_length,
int* input_end,
int* is_uncompressed) {
int size_nibbles;
int i;
- *input_end = BrotliReadBits(br, 1);
+ *input_end = (int)BrotliReadBits(br, 1);
*meta_block_length = 0;
*is_uncompressed = 0;
if (*input_end && BrotliReadBits(br, 1)) {
return;
}
- size_nibbles = BrotliReadBits(br, 2) + 4;
+ size_nibbles = (int)BrotliReadBits(br, 2) + 4;
for (i = 0; i < size_nibbles; ++i) {
- *meta_block_length |= BrotliReadBits(br, 4) << (i * 4);
+ *meta_block_length |= (int)BrotliReadBits(br, 4) << (i * 4);
}
++(*meta_block_length);
if (!*input_end) {
- *is_uncompressed = BrotliReadBits(br, 1);
+ *is_uncompressed = (int)BrotliReadBits(br, 1);
}
}
-// Decodes the next Huffman code from bit-stream.
+/* Decodes the next Huffman code from bit-stream. */
static BROTLI_INLINE int ReadSymbol(const HuffmanTree* tree,
BrotliBitReader* br) {
uint32_t bits;
- int bitpos;
+ uint32_t bitpos;
int lut_ix;
- int lut_bits;
+ uint8_t lut_bits;
const HuffmanTreeNode* node = tree->root_;
BrotliFillBitWindow(br);
bits = BrotliPrefetchBits(br);
bitpos = br->bit_pos_;
- // Check if we find the bit combination from the Huffman lookup table.
+ /* Check if we find the bit combination from the Huffman lookup table. */
lut_ix = bits & (HUFF_LUT - 1);
lut_bits = tree->lut_bits_[lut_ix];
if (lut_bits <= HUFF_LUT_BITS) {
@@ -124,7 +125,7 @@ static BROTLI_INLINE int ReadSymbol(const HuffmanTree* tree,
bitpos += HUFF_LUT_BITS;
bits >>= HUFF_LUT_BITS;
- // Decode the value from a binary tree.
+ /* Decode the value from a binary tree. */
assert(node != NULL);
do {
node = HuffmanTreeNextNode(node, bits & 1);
@@ -146,11 +147,10 @@ static int ReadHuffmanCodeLengths(
BrotliBitReader* br) {
int ok = 0;
int symbol;
- int max_symbol;
- int decode_number_of_code_length_codes;
- int prev_code_len = kDefaultCodeLength;
+ uint8_t prev_code_len = kDefaultCodeLength;
int repeat = 0;
- int repeat_length = 0;
+ uint8_t repeat_length = 0;
+ int space = 32768;
HuffmanTree tree;
if (!BrotliHuffmanTreeBuildImplicit(&tree, code_length_code_lengths,
@@ -164,33 +164,15 @@ static int ReadHuffmanCodeLengths(
printf("[ReadHuffmanCodeLengths] Unexpected end of input.\n");
return 0;
}
- decode_number_of_code_length_codes = BrotliReadBits(br, 1);
- BROTLI_LOG_UINT(decode_number_of_code_length_codes);
- if (decode_number_of_code_length_codes) {
- if (BrotliReadBits(br, 1)) {
- max_symbol = 68 + BrotliReadBits(br, 7);
- } else {
- max_symbol = 4 + BrotliReadBits(br, 6);
- }
- if (max_symbol > num_symbols) {
- printf("[ReadHuffmanCodeLengths] max_symbol > num_symbols (%d vs %d)\n",
- max_symbol, num_symbols);
- goto End;
- }
- } else {
- max_symbol = num_symbols;
- }
- BROTLI_LOG_UINT(max_symbol);
symbol = 0;
- while (symbol + repeat < num_symbols) {
- int code_len;
- if (max_symbol-- == 0) break;
+ while (symbol + repeat < num_symbols && space > 0) {
+ uint8_t code_len;
if (!BrotliReadMoreInput(br)) {
printf("[ReadHuffmanCodeLengths] Unexpected end of input.\n");
goto End;
}
- code_len = ReadSymbol(&tree, br);
+ code_len = (uint8_t)ReadSymbol(&tree, br);
BROTLI_LOG_UINT(symbol);
BROTLI_LOG_UINT(repeat);
BROTLI_LOG_UINT(repeat_length);
@@ -205,17 +187,35 @@ static int ReadHuffmanCodeLengths(
}
if (code_len < kCodeLengthRepeatCode) {
code_lengths[symbol++] = code_len;
- if (code_len != 0) prev_code_len = code_len;
+ if (code_len != 0) {
+ prev_code_len = code_len;
+ space -= 32768 >> code_len;
+ }
} else {
const int extra_bits = code_len - 14;
+ int i = repeat;
if (repeat > 0) {
repeat -= 2;
repeat <<= extra_bits;
}
- repeat += BrotliReadBits(br, extra_bits) + 3;
- repeat_length = (code_len == kCodeLengthRepeatCode ? prev_code_len : 0);
+ repeat += (int)BrotliReadBits(br, extra_bits) + 3;
+ if (repeat + symbol > num_symbols) {
+ goto End;
+ }
+ if (code_len == kCodeLengthRepeatCode) {
+ repeat_length = prev_code_len;
+ for (; i < repeat; ++i) {
+ space -= 32768 >> repeat_length;
+ }
+ } else {
+ repeat_length = 0;
+ }
}
}
+ if (space != 0) {
+ printf("[ReadHuffmanCodeLengths] space = %d\n", space);
+ goto End;
+ }
if (symbol + repeat > num_symbols) {
printf("[ReadHuffmanCodeLengths] symbol + repeat > num_symbols "
"(%d + %d vs %d)\n", symbol, repeat, num_symbols);
@@ -234,7 +234,7 @@ static int ReadHuffmanCode(int alphabet_size,
HuffmanTree* tree,
BrotliBitReader* br) {
int ok = 1;
- int simple_code;
+ int simple_code_or_skip;
uint8_t* code_lengths = NULL;
code_lengths =
@@ -247,21 +247,25 @@ static int ReadHuffmanCode(int alphabet_size,
printf("[ReadHuffmanCode] Unexpected end of input.\n");
return 0;
}
- simple_code = BrotliReadBits(br, 1);
- BROTLI_LOG_UINT(simple_code);
- if (simple_code) { // Read symbols, codes & code lengths directly.
+ /* simple_code_or_skip is used as follows:
+ 1 for simple code;
+ 0 for no skipping, 2 skips 2 code lengths, 3 skips 3 code lengths */
+ simple_code_or_skip = (int)BrotliReadBits(br, 2);
+ BROTLI_LOG_UINT(simple_code_or_skip);
+ if (simple_code_or_skip == 1) {
+ /* Read symbols, codes & code lengths directly. */
int i;
int max_bits_counter = alphabet_size - 1;
int max_bits = 0;
int symbols[4] = { 0 };
- const int num_symbols = BrotliReadBits(br, 2) + 1;
+ const int num_symbols = (int)BrotliReadBits(br, 2) + 1;
while (max_bits_counter) {
max_bits_counter >>= 1;
++max_bits;
}
- memset(code_lengths, 0, alphabet_size);
+ memset(code_lengths, 0, (size_t)alphabet_size);
for (i = 0; i < num_symbols; ++i) {
- symbols[i] = BrotliReadBits(br, max_bits);
+ symbols[i] = (int)BrotliReadBits(br, max_bits) % alphabet_size;
code_lengths[symbols[i]] = 2;
}
code_lengths[symbols[0]] = 1;
@@ -282,23 +286,20 @@ static int ReadHuffmanCode(int alphabet_size,
break;
}
BROTLI_LOG_UINT(num_symbols);
- } else { // Decode Huffman-coded code lengths.
+ } else { /* Decode Huffman-coded code lengths. */
int i;
uint8_t code_length_code_lengths[CODE_LENGTH_CODES] = { 0 };
- const int num_codes = BrotliReadBits(br, 4) + 3;
- BROTLI_LOG_UINT(num_codes);
- if (num_codes > CODE_LENGTH_CODES) {
- return 0;
- }
- for (i = BrotliReadBits(br, 1) * 2; i < num_codes; ++i) {
+ int space = 32;
+ for (i = simple_code_or_skip;
+ i < CODE_LENGTH_CODES && space > 0; ++i) {
int code_len_idx = kCodeLengthCodeOrder[i];
- int v = BrotliReadBits(br, 2);
+ uint8_t v = (uint8_t)BrotliReadBits(br, 2);
if (v == 1) {
- v = BrotliReadBits(br, 1);
+ v = (uint8_t)BrotliReadBits(br, 1);
if (v == 0) {
v = 2;
} else {
- v = BrotliReadBits(br, 1);
+ v = (uint8_t)BrotliReadBits(br, 1);
if (v == 0) {
v = 1;
} else {
@@ -310,6 +311,9 @@ static int ReadHuffmanCode(int alphabet_size,
}
code_length_code_lengths[code_len_idx] = v;
BROTLI_LOG_ARRAY_INDEX(code_length_code_lengths, code_len_idx);
+ if (v != 0) {
+ space -= (32 >> v);
+ }
}
ok = ReadHuffmanCodeLengths(code_length_code_lengths, alphabet_size,
code_lengths, br);
@@ -328,7 +332,7 @@ static int ReadHuffmanCode(int alphabet_size,
static int ReadCopyDistance(const HuffmanTree* tree,
int num_direct_codes,
int postfix_bits,
- uint32_t postfix_mask,
+ int postfix_mask,
BrotliBitReader* br) {
int code;
int nbits;
@@ -344,7 +348,7 @@ static int ReadCopyDistance(const HuffmanTree* tree,
nbits = (code >> 1) + 1;
offset = ((2 + (code & 1)) << nbits) - 4;
return (num_direct_codes +
- ((offset + BrotliReadBits(br, nbits)) << postfix_bits) +
+ ((offset + (int)BrotliReadBits(br, nbits)) << postfix_bits) +
postfix);
}
@@ -353,7 +357,7 @@ static int ReadBlockLength(const HuffmanTree* tree, BrotliBitReader* br) {
int nbits;
code = ReadSymbol(tree, br);
nbits = kBlockLengthPrefixCode[code].nbits;
- return kBlockLengthPrefixCode[code].offset + BrotliReadBits(br, nbits);
+ return kBlockLengthPrefixCode[code].offset + (int)BrotliReadBits(br, nbits);
}
static void ReadInsertAndCopy(const HuffmanTree* tree,
@@ -380,16 +384,16 @@ static void ReadInsertAndCopy(const HuffmanTree* tree,
*insert_len = kInsertLengthPrefixCode[insert_code].offset;
insert_extra_bits = kInsertLengthPrefixCode[insert_code].nbits;
if (insert_extra_bits > 0) {
- *insert_len += BrotliReadBits(br, insert_extra_bits);
+ *insert_len += (int)BrotliReadBits(br, insert_extra_bits);
}
*copy_len = kCopyLengthPrefixCode[copy_code].offset;
copy_extra_bits = kCopyLengthPrefixCode[copy_code].nbits;
if (copy_extra_bits > 0) {
- *copy_len += BrotliReadBits(br, copy_extra_bits);
+ *copy_len += (int)BrotliReadBits(br, copy_extra_bits);
}
}
-static int TranslateShortCodes(int code, int* ringbuffer, size_t index) {
+static int TranslateShortCodes(int code, int* ringbuffer, int index) {
int val;
if (code < NUM_DISTANCE_SHORT_CODES) {
index += kDistanceShortCodeIndexOffset[code];
@@ -412,7 +416,7 @@ static void InverseMoveToFrontTransform(uint8_t* v, int v_len) {
uint8_t mtf[256];
int i;
for (i = 0; i < 256; ++i) {
- mtf[i] = i;
+ mtf[i] = (uint8_t)i;
}
for (i = 0; i < v_len; ++i) {
uint8_t index = v[i];
@@ -421,7 +425,7 @@ static void InverseMoveToFrontTransform(uint8_t* v, int v_len) {
}
}
-// Contains a collection of huffman trees with the same alphabet size.
+/* Contains a collection of huffman trees with the same alphabet size. */
typedef struct {
int alphabet_size;
int num_htrees;
@@ -430,9 +434,13 @@ typedef struct {
static void HuffmanTreeGroupInit(HuffmanTreeGroup* group, int alphabet_size,
int ntrees) {
+ int i;
group->alphabet_size = alphabet_size;
group->num_htrees = ntrees;
- group->htrees = (HuffmanTree*)malloc(sizeof(HuffmanTree) * ntrees);
+ group->htrees = (HuffmanTree*)malloc(sizeof(HuffmanTree) * (size_t)ntrees);
+ for (i = 0; i < ntrees; ++i) {
+ group->htrees[i].root_ = NULL;
+ }
}
static void HuffmanTreeGroupRelease(HuffmanTreeGroup* group) {
@@ -440,7 +448,9 @@ static void HuffmanTreeGroupRelease(HuffmanTreeGroup* group) {
for (i = 0; i < group->num_htrees; ++i) {
BrotliHuffmanTreeRelease(&group->htrees[i]);
}
- free(group->htrees);
+ if (group->htrees) {
+ free(group->htrees);
+ }
}
static int HuffmanTreeGroupDecode(HuffmanTreeGroup* group,
@@ -468,19 +478,22 @@ static int DecodeContextMap(int context_map_size,
BROTLI_LOG_UINT(context_map_size);
BROTLI_LOG_UINT(*num_htrees);
- *context_map = (uint8_t*)malloc(context_map_size);
+ *context_map = (uint8_t*)malloc((size_t)context_map_size);
+ if (*context_map == 0) {
+ return 0;
+ }
if (*num_htrees <= 1) {
- memset(*context_map, 0, context_map_size);
+ memset(*context_map, 0, (size_t)context_map_size);
return 1;
}
{
HuffmanTree tree_index_htree;
- int use_rle_for_zeros = BrotliReadBits(br, 1);
+ int use_rle_for_zeros = (int)BrotliReadBits(br, 1);
int max_run_length_prefix = 0;
int i;
if (use_rle_for_zeros) {
- max_run_length_prefix = BrotliReadBits(br, 4) + 1;
+ max_run_length_prefix = (int)BrotliReadBits(br, 4) + 1;
}
if (!ReadHuffmanCode(*num_htrees + max_run_length_prefix,
&tree_index_htree, br)) {
@@ -498,13 +511,17 @@ static int DecodeContextMap(int context_map_size,
(*context_map)[i] = 0;
++i;
} else if (code <= max_run_length_prefix) {
- int reps = 1 + (1 << code) + BrotliReadBits(br, code);
+ int reps = 1 + (1 << code) + (int)BrotliReadBits(br, code);
while (--reps) {
+ if (i >= context_map_size) {
+ ok = 0;
+ goto End;
+ }
(*context_map)[i] = 0;
++i;
}
} else {
- (*context_map)[i] = code - max_run_length_prefix;
+ (*context_map)[i] = (uint8_t)(code - max_run_length_prefix);
++i;
}
}
@@ -517,14 +534,15 @@ static int DecodeContextMap(int context_map_size,
return ok;
}
-static BROTLI_INLINE void DecodeBlockType(const HuffmanTree* trees,
- int tree_type,
- int* block_types,
- int* ringbuffers,
- size_t* indexes,
- BrotliBitReader* br) {
+static BROTLI_INLINE void DecodeBlockType(const int max_block_type,
+ const HuffmanTree* trees,
+ int tree_type,
+ int* block_types,
+ int* ringbuffers,
+ int* indexes,
+ BrotliBitReader* br) {
int* ringbuffer = ringbuffers + tree_type * 2;
- size_t* index = indexes + tree_type;
+ int* index = indexes + tree_type;
int type_code = ReadSymbol(trees + tree_type, br);
int block_type;
if (type_code == 0) {
@@ -534,47 +552,51 @@ static BROTLI_INLINE void DecodeBlockType(const HuffmanTree* trees,
} else {
block_type = type_code - 2;
}
+ if (block_type >= max_block_type) {
+ block_type -= max_block_type;
+ }
block_types[tree_type] = block_type;
ringbuffer[(*index) & 1] = block_type;
++(*index);
}
-// Copy len bytes from src to dst. It can write up to ten extra bytes
-// after the end of the copy.
-//
-// The main part of this loop is a simple copy of eight bytes at a time until
-// we've copied (at least) the requested amount of bytes. However, if dst and
-// src are less than eight bytes apart (indicating a repeating pattern of
-// length < 8), we first need to expand the pattern in order to get the correct
-// results. For instance, if the buffer looks like this, with the eight-byte
-// <src> and <dst> patterns marked as intervals:
-//
-// abxxxxxxxxxxxx
-// [------] src
-// [------] dst
-//
-// a single eight-byte copy from <src> to <dst> will repeat the pattern once,
-// after which we can move <dst> two bytes without moving <src>:
-//
-// ababxxxxxxxxxx
-// [------] src
-// [------] dst
-//
-// and repeat the exercise until the two no longer overlap.
-//
-// This allows us to do very well in the special case of one single byte
-// repeated many times, without taking a big hit for more general cases.
-//
-// The worst case of extra writing past the end of the match occurs when
-// dst - src == 1 and len == 1; the last copy will read from byte positions
-// [0..7] and write to [4..11], whereas it was only supposed to write to
-// position 1. Thus, ten excess bytes.
+/* Copy len bytes from src to dst. It can write up to ten extra bytes
+ after the end of the copy.
+
+ The main part of this loop is a simple copy of eight bytes at a time until
+ we've copied (at least) the requested amount of bytes. However, if dst and
+ src are less than eight bytes apart (indicating a repeating pattern of
+ length < 8), we first need to expand the pattern in order to get the correct
+ results. For instance, if the buffer looks like this, with the eight-byte
+ <src> and <dst> patterns marked as intervals:
+
+ abxxxxxxxxxxxx
+ [------] src
+ [------] dst
+
+ a single eight-byte copy from <src> to <dst> will repeat the pattern once,
+ after which we can move <dst> two bytes without moving <src>:
+
+ ababxxxxxxxxxx
+ [------] src
+ [------] dst
+
+ and repeat the exercise until the two no longer overlap.
+
+ This allows us to do very well in the special case of one single byte
+ repeated many times, without taking a big hit for more general cases.
+
+ The worst case of extra writing past the end of the match occurs when
+ dst - src == 1 and len == 1; the last copy will read from byte positions
+ [0..7] and write to [4..11], whereas it was only supposed to write to
+ position 1. Thus, ten excess bytes.
+*/
static BROTLI_INLINE void IncrementalCopyFastPath(
uint8_t* dst, const uint8_t* src, int len) {
if (src < dst) {
while (dst - src < 8) {
UNALIGNED_COPY64(dst, src);
- len -= dst - src;
+ len -= (int)(dst - src);
dst += dst - src;
}
}
@@ -592,7 +614,7 @@ int BrotliDecompressedSize(size_t encoded_size,
BrotliMemInput memin;
BrotliInput input = BrotliInitMemInput(encoded_buffer, encoded_size, &memin);
BrotliBitReader br;
- size_t meta_block_len;
+ int meta_block_len;
int input_end;
int is_uncompressed;
if (!BrotliInitBitReader(&br, input)) {
@@ -603,7 +625,7 @@ int BrotliDecompressedSize(size_t encoded_size,
if (!input_end) {
return 0;
}
- *decoded_size = meta_block_len;
+ *decoded_size = (size_t)meta_block_len;
return 1;
}
@@ -623,25 +645,28 @@ int BrotliDecompressBuffer(size_t encoded_size,
int BrotliDecompress(BrotliInput input, BrotliOutput output) {
int ok = 1;
int i;
- size_t pos = 0;
+ int pos = 0;
int input_end = 0;
int window_bits = 0;
- size_t max_backward_distance;
- size_t ringbuffer_size;
- size_t ringbuffer_mask;
+ int max_backward_distance;
+ int max_distance = 0;
+ int ringbuffer_size;
+ int ringbuffer_mask;
uint8_t* ringbuffer;
uint8_t* ringbuffer_end;
- // This ring buffer holds a few past copy distances that will be used by
- // some special distance codes.
+ /* This ring buffer holds a few past copy distances that will be used by */
+ /* some special distance codes. */
int dist_rb[4] = { 16, 15, 11, 4 };
- size_t dist_rb_idx = 0;
- // The previous 2 bytes used for context.
+ int dist_rb_idx = 0;
+ /* The previous 2 bytes used for context. */
uint8_t prev_byte1 = 0;
uint8_t prev_byte2 = 0;
HuffmanTreeGroup hgroup[3];
BrotliBitReader br;
- static const int kRingBufferWriteAheadSlack = 16;
+ /* 16 bytes would be enough, but we add some more slack for transforms */
+ /* to work at the end of the ringbuffer. */
+ static const int kRingBufferWriteAheadSlack = 128;
static const int kMaxDictionaryWordLength = 0;
@@ -649,31 +674,33 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
return 0;
}
- // Decode window size.
+ /* Decode window size. */
window_bits = DecodeWindowBits(&br);
- max_backward_distance = (1ULL << window_bits) - 16;
+ max_backward_distance = (1 << window_bits) - 16;
- ringbuffer_size = 1ULL << window_bits;
+ ringbuffer_size = 1 << window_bits;
ringbuffer_mask = ringbuffer_size - 1;
- ringbuffer = (uint8_t*)malloc(ringbuffer_size +
- kRingBufferWriteAheadSlack +
- kMaxDictionaryWordLength);
+ ringbuffer = (uint8_t*)malloc((size_t)(ringbuffer_size +
+ kRingBufferWriteAheadSlack +
+ kMaxDictionaryWordLength));
+ if (!ringbuffer) {
+ ok = 0;
+ }
ringbuffer_end = ringbuffer + ringbuffer_size;
while (!input_end && ok) {
- size_t meta_block_len = 0;
- size_t meta_block_end_pos;
+ int meta_block_remaining_len = 0;
int is_uncompressed;
- uint32_t block_length[3] = { 1 << 28, 1 << 28, 1 << 28 };
+ int block_length[3] = { 1 << 28, 1 << 28, 1 << 28 };
int block_type[3] = { 0 };
int num_block_types[3] = { 1, 1, 1 };
int block_type_rb[6] = { 0, 1, 0, 1, 0, 1 };
- size_t block_type_rb_index[3] = { 0 };
+ int block_type_rb_index[3] = { 0 };
HuffmanTree block_type_trees[3];
HuffmanTree block_len_trees[3];
int distance_postfix_bits;
int num_direct_distance_codes;
- uint32_t distance_postfix_mask;
+ int distance_postfix_mask;
int num_distance_codes;
uint8_t* context_map = NULL;
uint8_t* context_modes = NULL;
@@ -703,22 +730,24 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
goto End;
}
BROTLI_LOG_UINT(pos);
- DecodeMetaBlockLength(&br, &meta_block_len, &input_end, &is_uncompressed);
- BROTLI_LOG_UINT(meta_block_len);
- if (meta_block_len == 0) {
+ DecodeMetaBlockLength(&br, &meta_block_remaining_len,
+ &input_end, &is_uncompressed);
+ BROTLI_LOG_UINT(meta_block_remaining_len);
+ if (meta_block_remaining_len == 0) {
goto End;
}
- meta_block_end_pos = pos + meta_block_len;
if (is_uncompressed) {
- BrotliSetBitPos(&br, (br.bit_pos_ + 7) & ~7);
- for (; pos < meta_block_end_pos; ++pos) {
- ringbuffer[pos & ringbuffer_mask] = BrotliReadBits(&br, 8);
+ BrotliSetBitPos(&br, (br.bit_pos_ + 7) & (uint32_t)(~7UL));
+ while (meta_block_remaining_len) {
+ ringbuffer[pos & ringbuffer_mask] = (uint8_t)BrotliReadBits(&br, 8);
if ((pos & ringbuffer_mask) == ringbuffer_mask) {
- if (BrotliWrite(output, ringbuffer, ringbuffer_size) < 0) {
+ if (BrotliWrite(output, ringbuffer, (size_t)ringbuffer_size) < 0) {
ok = 0;
goto End;
}
}
+ ++pos;
+ --meta_block_remaining_len;
}
goto End;
}
@@ -750,15 +779,19 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
ok = 0;
goto End;
}
- distance_postfix_bits = BrotliReadBits(&br, 2);
+ distance_postfix_bits = (int)BrotliReadBits(&br, 2);
num_direct_distance_codes = NUM_DISTANCE_SHORT_CODES +
- (BrotliReadBits(&br, 4) << distance_postfix_bits);
+ ((int)BrotliReadBits(&br, 4) << distance_postfix_bits);
distance_postfix_mask = (1 << distance_postfix_bits) - 1;
num_distance_codes = (num_direct_distance_codes +
(48 << distance_postfix_bits));
- context_modes = (uint8_t*)malloc(num_block_types[0]);
+ context_modes = (uint8_t*)malloc((size_t)num_block_types[0]);
+ if (context_modes == 0) {
+ ok = 0;
+ goto End;
+ }
for (i = 0; i < num_block_types[0]; ++i) {
- context_modes[i] = BrotliReadBits(&br, 2) << 1;
+ context_modes[i] = (uint8_t)(BrotliReadBits(&br, 2) << 1);
BROTLI_LOG_ARRAY_INDEX(context_modes, i);
}
BROTLI_LOG_UINT(num_direct_distance_codes);
@@ -790,12 +823,11 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
context_lookup_offset1 = kContextLookupOffsets[context_mode];
context_lookup_offset2 = kContextLookupOffsets[context_mode + 1];
- while (pos < meta_block_end_pos) {
+ while (meta_block_remaining_len > 0) {
int insert_length;
int copy_length;
int distance_code;
int distance;
- size_t max_distance;
uint8_t context;
int j;
const uint8_t* copy_src;
@@ -806,7 +838,8 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
goto End;
}
if (block_length[1] == 0) {
- DecodeBlockType(block_type_trees, 1, block_type, block_type_rb,
+ DecodeBlockType(num_block_types[1],
+ block_type_trees, 1, block_type, block_type_rb,
block_type_rb_index, &br);
block_length[1] = ReadBlockLength(&block_len_trees[1], &br);
}
@@ -823,7 +856,8 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
goto End;
}
if (block_length[0] == 0) {
- DecodeBlockType(block_type_trees, 0, block_type, block_type_rb,
+ DecodeBlockType(num_block_types[0],
+ block_type_trees, 0, block_type, block_type_rb,
block_type_rb_index, &br);
block_length[0] = ReadBlockLength(&block_len_trees[0], &br);
context_offset = block_type[0] << kLiteralContextBits;
@@ -838,19 +872,21 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
literal_htree_index = context_map_slice[context];
--block_length[0];
prev_byte2 = prev_byte1;
- prev_byte1 = ReadSymbol(&hgroup[0].htrees[literal_htree_index], &br);
+ prev_byte1 = (uint8_t)ReadSymbol(&hgroup[0].htrees[literal_htree_index],
+ &br);
ringbuffer[pos & ringbuffer_mask] = prev_byte1;
BROTLI_LOG_UINT(literal_htree_index);
BROTLI_LOG_ARRAY_INDEX(ringbuffer, pos & ringbuffer_mask);
if ((pos & ringbuffer_mask) == ringbuffer_mask) {
- if (BrotliWrite(output, ringbuffer, ringbuffer_size) < 0) {
+ if (BrotliWrite(output, ringbuffer, (size_t)ringbuffer_size) < 0) {
ok = 0;
goto End;
}
}
++pos;
}
- if (pos == meta_block_end_pos) break;
+ meta_block_remaining_len -= insert_length;
+ if (meta_block_remaining_len <= 0) break;
if (distance_code < 0) {
uint8_t context;
@@ -860,15 +896,16 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
goto End;
}
if (block_length[2] == 0) {
- DecodeBlockType(block_type_trees, 2, block_type, block_type_rb,
+ DecodeBlockType(num_block_types[2],
+ block_type_trees, 2, block_type, block_type_rb,
block_type_rb_index, &br);
block_length[2] = ReadBlockLength(&block_len_trees[2], &br);
- dist_htree_index = block_type[2];
+ dist_htree_index = (uint8_t)block_type[2];
dist_context_offset = block_type[2] << kDistanceContextBits;
dist_context_map_slice = dist_context_map + dist_context_offset;
}
--block_length[2];
- context = copy_length > 4 ? 3 : copy_length - 2;
+ context = (uint8_t)(copy_length > 4 ? 3 : copy_length - 2);
dist_htree_index = dist_context_map_slice[context];
distance_code = ReadCopyDistance(&hgroup[2].htrees[dist_htree_index],
num_direct_distance_codes,
@@ -877,33 +914,39 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
&br);
}
- // Convert the distance code to the actual distance by possibly looking
- // up past distnaces from the ringbuffer.
+ /* Convert the distance code to the actual distance by possibly looking */
+ /* up past distnaces from the ringbuffer. */
distance = TranslateShortCodes(distance_code, dist_rb, dist_rb_idx);
+ if (distance < 0) {
+ ok = 0;
+ goto End;
+ }
if (distance_code > 0) {
dist_rb[dist_rb_idx & 3] = distance;
++dist_rb_idx;
}
BROTLI_LOG_UINT(distance);
- max_distance = max_backward_distance;
- if (pos < max_distance) {
+ if (pos < max_backward_distance &&
+ max_distance != max_backward_distance) {
max_distance = pos;
+ } else {
+ max_distance = max_backward_distance;
}
copy_dst = &ringbuffer[pos & ringbuffer_mask];
- if ((size_t)distance > max_distance) {
- printf("Invalid backward reference. pos: %lu distance: %d "
- "len: %d end: %lu\n", (unsigned long)pos, distance, copy_length,
- (unsigned long)meta_block_end_pos);
+ if (distance > max_distance) {
+ printf("Invalid backward reference. pos: %d distance: %d "
+ "len: %d bytes left: %d\n", pos, distance, copy_length,
+ meta_block_remaining_len);
ok = 0;
goto End;
} else {
- if (pos + copy_length > meta_block_end_pos) {
- printf("Invalid backward reference. pos: %lu distance: %d "
- "len: %d end: %lu\n", (unsigned long)pos, distance,
- copy_length, (unsigned long)meta_block_end_pos);
+ if (copy_length > meta_block_remaining_len) {
+ printf("Invalid backward reference. pos: %d distance: %d "
+ "len: %d bytes left: %d\n", pos, distance, copy_length,
+ meta_block_remaining_len);
ok = 0;
goto End;
}
@@ -920,6 +963,7 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
IncrementalCopyFastPath(copy_dst, copy_src, copy_length);
}
pos += copy_length;
+ meta_block_remaining_len -= copy_length;
copy_length = 0;
}
#endif
@@ -928,25 +972,36 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
ringbuffer[pos & ringbuffer_mask] =
ringbuffer[(pos - distance) & ringbuffer_mask];
if ((pos & ringbuffer_mask) == ringbuffer_mask) {
- if (BrotliWrite(output, ringbuffer, ringbuffer_size) < 0) {
+ if (BrotliWrite(output, ringbuffer, (size_t)ringbuffer_size) < 0) {
ok = 0;
goto End;
}
}
++pos;
+ --meta_block_remaining_len;
}
}
- // When we get here, we must have inserted at least one literal and made
- // a copy of at least length two, therefore accessing the last 2 bytes is
- // valid.
+ /* When we get here, we must have inserted at least one literal and */
+ /* made a copy of at least length two, therefore accessing the last 2 */
+ /* bytes is valid. */
prev_byte1 = ringbuffer[(pos - 1) & ringbuffer_mask];
prev_byte2 = ringbuffer[(pos - 2) & ringbuffer_mask];
}
+
+ /* Protect pos from overflow, wrap it around at every GB of input data */
+ pos &= 0x3fffffff;
+
End:
- free(context_modes);
- free(context_map);
- free(dist_context_map);
+ if (context_modes != 0) {
+ free(context_modes);
+ }
+ if (context_map != 0) {
+ free(context_map);
+ }
+ if (dist_context_map != 0) {
+ free(dist_context_map);
+ }
for (i = 0; i < 3; ++i) {
HuffmanTreeGroupRelease(&hgroup[i]);
BrotliHuffmanTreeRelease(&block_type_trees[i]);
@@ -954,13 +1009,15 @@ int BrotliDecompress(BrotliInput input, BrotliOutput output) {
}
}
- if (BrotliWrite(output, ringbuffer, pos & ringbuffer_mask) < 0) {
- ok = 0;
+ if (ringbuffer != 0) {
+ if (BrotliWrite(output, ringbuffer, (size_t)(pos & ringbuffer_mask)) < 0) {
+ ok = 0;
+ }
+ free(ringbuffer);
}
- free(ringbuffer);
return ok;
}
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
diff --git a/brotli/dec/decode.h b/brotli/dec/decode.h
index 760ec79..9182438 100644
--- a/brotli/dec/decode.h
+++ b/brotli/dec/decode.h
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// API for Brotli decompression
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ API for Brotli decompression
+*/
#ifndef BROTLI_DEC_DECODE_H_
#define BROTLI_DEC_DECODE_H_
@@ -24,28 +25,28 @@
extern "C" {
#endif
-// Sets *decoded_size to the decompressed size of the given encoded stream.
-// Returns 1 on success, 0 on failure.
+/* Sets *decoded_size to the decompressed size of the given encoded stream. */
+/* Returns 1 on success, 0 on failure. */
int BrotliDecompressedSize(size_t encoded_size,
const uint8_t* encoded_buffer,
size_t* decoded_size);
-// Decompresses the data in encoded_buffer into decoded_buffer, and sets
-// *decoded_size to the decompressed length.
-// Returns 0 if there was either a bit stream error or memory allocation error,
-// and 1 otherwise.
-// If decoded size is zero, returns 1 and keeps decoded_buffer unchanged.
+/* Decompresses the data in encoded_buffer into decoded_buffer, and sets */
+/* *decoded_size to the decompressed length. */
+/* Returns 0 if there was either a bit stream error or memory allocation */
+/* error, and 1 otherwise. */
+/* If decoded size is zero, returns 1 and keeps decoded_buffer unchanged. */
int BrotliDecompressBuffer(size_t encoded_size,
const uint8_t* encoded_buffer,
size_t* decoded_size,
uint8_t* decoded_buffer);
-// Same as above, but uses the specified input and output callbacks instead of
-// reading from and writing to pre-allocated memory buffers.
+/* Same as above, but uses the specified input and output callbacks instead */
+/* of reading from and writing to pre-allocated memory buffers. */
int BrotliDecompress(BrotliInput input, BrotliOutput output);
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
-#endif // BROTLI_DEC_DECODE_H_
+#endif /* BROTLI_DEC_DECODE_H_ */
diff --git a/brotli/dec/huffman.c b/brotli/dec/huffman.c
index 6327792..20e6223 100644
--- a/brotli/dec/huffman.c
+++ b/brotli/dec/huffman.c
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Utilities for building and looking up Huffman trees.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Utilities for building and looking up Huffman trees.
+*/
#include <assert.h>
#include <stdlib.h>
@@ -28,7 +29,7 @@ extern "C" {
#define MAX_ALLOWED_CODE_LENGTH 15
static void TreeNodeInit(HuffmanTreeNode* const node) {
- node->children_ = -1; // means: 'unassigned so far'
+ node->children_ = -1; /* means: 'unassigned so far' */
}
static int NodeIsEmpty(const HuffmanTreeNode* const node) {
@@ -51,16 +52,17 @@ static void AssignChildren(HuffmanTree* const tree,
static int TreeInit(HuffmanTree* const tree, int num_leaves) {
assert(tree != NULL);
+ tree->root_ = NULL;
if (num_leaves == 0) return 0;
- // We allocate maximum possible nodes in the tree at once.
- // Note that a Huffman tree is a full binary tree; and in a full binary tree
- // with L leaves, the total number of nodes N = 2 * L - 1.
+ /* We allocate maximum possible nodes in the tree at once. */
+ /* Note that a Huffman tree is a full binary tree; and in a full binary */
+ /* tree with L leaves, the total number of nodes N = 2 * L - 1. */
tree->max_nodes_ = 2 * num_leaves - 1;
- assert(tree->max_nodes_ < (1 << 16)); // limit for the lut_jump_ table
+ assert(tree->max_nodes_ < (1 << 16)); /* limit for the lut_jump_ table */
tree->root_ = (HuffmanTreeNode*)BrotliSafeMalloc((uint64_t)tree->max_nodes_,
sizeof(*tree->root_));
if (tree->root_ == NULL) return 0;
- TreeNodeInit(tree->root_); // Initialize root.
+ TreeNodeInit(tree->root_); /* Initialize root. */
tree->num_nodes_ = 1;
memset(tree->lut_bits_, 255, sizeof(tree->lut_bits_));
memset(tree->lut_jump_, 0, sizeof(tree->lut_jump_));
@@ -69,16 +71,18 @@ static int TreeInit(HuffmanTree* const tree, int num_leaves) {
void BrotliHuffmanTreeRelease(HuffmanTree* const tree) {
if (tree != NULL) {
- free(tree->root_);
+ if (tree->root_ != NULL) {
+ free(tree->root_);
+ }
tree->root_ = NULL;
tree->max_nodes_ = 0;
tree->num_nodes_ = 0;
}
}
-// Utility: converts Huffman code lengths to corresponding Huffman codes.
-// 'huff_codes' should be pre-allocated.
-// Returns false in case of error (memory allocation, invalid codes).
+/* Utility: converts Huffman code lengths to corresponding Huffman codes. */
+/* 'huff_codes' should be pre-allocated. */
+/* Returns false in case of error (memory allocation, invalid codes). */
static int HuffmanCodeLengthsToCodes(const uint8_t* const code_lengths,
int code_lengths_size,
int* const huff_codes) {
@@ -93,7 +97,7 @@ static int HuffmanCodeLengthsToCodes(const uint8_t* const code_lengths,
assert(code_lengths_size > 0);
assert(huff_codes != NULL);
- // Calculate max code length.
+ /* Calculate max code length. */
for (symbol = 0; symbol < code_lengths_size; ++symbol) {
if (code_lengths[symbol] > max_code_length) {
max_code_length = code_lengths[symbol];
@@ -101,23 +105,24 @@ static int HuffmanCodeLengthsToCodes(const uint8_t* const code_lengths,
}
if (max_code_length > MAX_ALLOWED_CODE_LENGTH) return 0;
- // Calculate code length histogram.
+ /* Calculate code length histogram. */
for (symbol = 0; symbol < code_lengths_size; ++symbol) {
++code_length_hist[code_lengths[symbol]];
}
code_length_hist[0] = 0;
- // Calculate the initial values of 'next_codes' for each code length.
- // next_codes[code_len] denotes the code to be assigned to the next symbol
- // of code length 'code_len'.
+ /* Calculate the initial values of 'next_codes' for each code length. */
+ /* next_codes[code_len] denotes the code to be assigned to the next symbol */
+ /* of code length 'code_len'. */
curr_code = 0;
- next_codes[0] = -1; // Unused, as code length = 0 implies code doesn't exist.
+ next_codes[0] = -1; /* Unused, as code length = 0 implies */
+ /* code doesn't exist. */
for (code_len = 1; code_len <= max_code_length; ++code_len) {
curr_code = (curr_code + code_length_hist[code_len - 1]) << 1;
next_codes[code_len] = curr_code;
}
- // Get symbols.
+ /* Get symbols. */
for (symbol = 0; symbol < code_lengths_size; ++symbol) {
if (code_lengths[symbol] > 0) {
huff_codes[symbol] = next_codes[code_lengths[symbol]]++;
@@ -158,7 +163,7 @@ static int TreeAddSymbol(HuffmanTree* const tree,
--i;
idx = base_code | (i << code_length);
tree->lut_symbol_[idx] = (int16_t)symbol;
- tree->lut_bits_[idx] = code_length;
+ tree->lut_bits_[idx] = (uint8_t)code_length;
} while (i > 0);
} else {
base_code = ReverseBitsShort((code >> (code_length - HUFF_LUT_BITS)),
@@ -169,10 +174,10 @@ static int TreeAddSymbol(HuffmanTree* const tree,
return 0;
}
if (NodeIsEmpty(node)) {
- if (IsFull(tree)) return 0; // error: too many symbols.
+ if (IsFull(tree)) return 0; /* error: too many symbols. */
AssignChildren(tree, node);
} else if (!HuffmanTreeNodeIsNotLeaf(node)) {
- return 0; // leaf is already occupied.
+ return 0; /* leaf is already occupied. */
}
node += node->children_ + ((code >> code_length) & 1);
if (--step == 0) {
@@ -180,11 +185,11 @@ static int TreeAddSymbol(HuffmanTree* const tree,
}
}
if (NodeIsEmpty(node)) {
- node->children_ = 0; // turn newly created node into a leaf.
+ node->children_ = 0; /* turn newly created node into a leaf. */
} else if (HuffmanTreeNodeIsNotLeaf(node)) {
- return 0; // trying to assign a symbol to already used code.
+ return 0; /* trying to assign a symbol to already used code. */
}
- node->symbol_ = symbol; // Add symbol in this node.
+ node->symbol_ = symbol; /* Add symbol in this node. */
return 1;
}
@@ -198,30 +203,30 @@ int BrotliHuffmanTreeBuildImplicit(HuffmanTree* const tree,
assert(tree != NULL);
assert(code_lengths != NULL);
- // Find out number of symbols and the root symbol.
+ /* Find out number of symbols and the root symbol. */
for (symbol = 0; symbol < code_lengths_size; ++symbol) {
if (code_lengths[symbol] > 0) {
- // Note: code length = 0 indicates non-existent symbol.
+ /* Note: code length = 0 indicates non-existent symbol. */
++num_symbols;
root_symbol = symbol;
}
}
- // Initialize the tree. Will fail for num_symbols = 0
+ /* Initialize the tree. Will fail for num_symbols = 0 */
if (!TreeInit(tree, num_symbols)) return 0;
- // Build tree.
- if (num_symbols == 1) { // Trivial case.
+ /* Build tree. */
+ if (num_symbols == 1) { /* Trivial case. */
const int max_symbol = code_lengths_size;
if (root_symbol < 0 || root_symbol >= max_symbol) {
BrotliHuffmanTreeRelease(tree);
return 0;
}
return TreeAddSymbol(tree, root_symbol, 0, 0);
- } else { // Normal case.
+ } else { /* Normal case. */
int ok = 0;
- // Get Huffman codes from the code lengths.
+ /* Get Huffman codes from the code lengths. */
int* const codes =
(int*)BrotliSafeMalloc((uint64_t)code_lengths_size, sizeof(*codes));
if (codes == NULL) goto End;
@@ -230,7 +235,7 @@ int BrotliHuffmanTreeBuildImplicit(HuffmanTree* const tree,
goto End;
}
- // Add symbols one-by-one.
+ /* Add symbols one-by-one. */
for (symbol = 0; symbol < code_lengths_size; ++symbol) {
if (code_lengths[symbol] > 0) {
if (!TreeAddSymbol(tree, symbol, codes[symbol], code_lengths[symbol])) {
@@ -248,5 +253,5 @@ int BrotliHuffmanTreeBuildImplicit(HuffmanTree* const tree,
}
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
diff --git a/brotli/dec/huffman.h b/brotli/dec/huffman.h
index f1a671d..fbd0744 100644
--- a/brotli/dec/huffman.h
+++ b/brotli/dec/huffman.h
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Utilities for building and looking up Huffman trees.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Utilities for building and looking up Huffman trees.
+*/
#ifndef BROTLI_DEC_HUFFMAN_H_
#define BROTLI_DEC_HUFFMAN_H_
@@ -24,51 +25,51 @@
extern "C" {
#endif
-// A node of a Huffman tree.
+/* A node of a Huffman tree. */
typedef struct {
int symbol_;
- int children_; // delta offset to both children (contiguous) or 0 if leaf.
+ int children_; /* delta offset to both children (contiguous) or 0 if leaf. */
} HuffmanTreeNode;
-// Huffman Tree.
+/* Huffman Tree. */
#define HUFF_LUT_BITS 7
#define HUFF_LUT (1U << HUFF_LUT_BITS)
typedef struct HuffmanTree HuffmanTree;
struct HuffmanTree {
- // Fast lookup for short bit lengths.
+ /* Fast lookup for short bit lengths. */
uint8_t lut_bits_[HUFF_LUT];
int16_t lut_symbol_[HUFF_LUT];
int16_t lut_jump_[HUFF_LUT];
- // Complete tree for lookups.
- HuffmanTreeNode* root_; // all the nodes, starting at root.
- int max_nodes_; // max number of nodes
- int num_nodes_; // number of currently occupied nodes
+ /* Complete tree for lookups. */
+ HuffmanTreeNode* root_; /* all the nodes, starting at root. */
+ int max_nodes_; /* max number of nodes */
+ int num_nodes_; /* number of currently occupied nodes */
};
-// Returns true if the given node is not a leaf of the Huffman tree.
+/* Returns true if the given node is not a leaf of the Huffman tree. */
static BROTLI_INLINE int HuffmanTreeNodeIsNotLeaf(
const HuffmanTreeNode* const node) {
return node->children_;
}
-// Go down one level. Most critical function. 'right_child' must be 0 or 1.
+/* Go down one level. Most critical function. 'right_child' must be 0 or 1. */
static BROTLI_INLINE const HuffmanTreeNode* HuffmanTreeNextNode(
const HuffmanTreeNode* node, int right_child) {
return node + node->children_ + right_child;
}
-// Releases the nodes of the Huffman tree.
-// Note: It does NOT free 'tree' itself.
+/* Releases the nodes of the Huffman tree. */
+/* Note: It does NOT free 'tree' itself. */
void BrotliHuffmanTreeRelease(HuffmanTree* const tree);
-// Builds Huffman tree assuming code lengths are implicitly in symbol order.
-// Returns false in case of error (invalid tree or memory error).
+/* Builds Huffman tree assuming code lengths are implicitly in symbol order. */
+/* Returns false in case of error (invalid tree or memory error). */
int BrotliHuffmanTreeBuildImplicit(HuffmanTree* const tree,
const uint8_t* const code_lengths,
int code_lengths_size);
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
-#endif // BROTLI_DEC_HUFFMAN_H_
+#endif /* BROTLI_DEC_HUFFMAN_H_ */
diff --git a/brotli/dec/prefix.h b/brotli/dec/prefix.h
index 500bd10..06afe4d 100644
--- a/brotli/dec/prefix.h
+++ b/brotli/dec/prefix.h
@@ -1,25 +1,26 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Lookup tables to map prefix codes to value ranges. This is used during
-// decoding of the block lengths, literal insertion lengths and copy lengths.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Lookup tables to map prefix codes to value ranges. This is used during
+ decoding of the block lengths, literal insertion lengths and copy lengths.
+*/
#ifndef BROTLI_DEC_PREFIX_H_
#define BROTLI_DEC_PREFIX_H_
-// Represents the range of values belonging to a prefix code:
-// [offset, offset + 2^nbits)
+/* Represents the range of values belonging to a prefix code: */
+/* [offset, offset + 2^nbits) */
struct PrefixCodeRange {
int offset;
int nbits;
@@ -61,4 +62,4 @@ static const int kCopyRangeLut[9] = {
0, 8, 0, 8, 16, 0, 16, 8, 16,
};
-#endif // BROTLI_DEC_PREFIX_H_
+#endif /* BROTLI_DEC_PREFIX_H_ */
diff --git a/brotli/dec/safe_malloc.c b/brotli/dec/safe_malloc.c
index 41fa480..ef1624c 100644
--- a/brotli/dec/safe_malloc.c
+++ b/brotli/dec/safe_malloc.c
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Size-checked memory allocation.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Size-checked memory allocation.
+*/
#include <stdlib.h>
#include "./safe_malloc.h"
@@ -21,7 +22,7 @@
extern "C" {
#endif
-// Returns 0 in case of overflow of nmemb * size.
+/* Returns 0 in case of overflow of nmemb * size. */
static int CheckSizeArgumentsOverflow(uint64_t nmemb, size_t size) {
const uint64_t total_size = nmemb * size;
if (nmemb == 0) return 1;
@@ -37,5 +38,5 @@ void* BrotliSafeMalloc(uint64_t nmemb, size_t size) {
}
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
diff --git a/brotli/dec/safe_malloc.h b/brotli/dec/safe_malloc.h
index 5a334fc..9a73b0e 100644
--- a/brotli/dec/safe_malloc.h
+++ b/brotli/dec/safe_malloc.h
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Size-checked memory allocation.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Size-checked memory allocation.
+*/
#ifndef BROTLI_UTILS_UTILS_H_
#define BROTLI_UTILS_UTILS_H_
@@ -25,19 +26,20 @@
extern "C" {
#endif
-// This is the maximum memory amount that we will ever try to allocate.
-#define BROTLI_MAX_ALLOCABLE_MEMORY (1ULL << 40)
+/* This is the maximum memory amount that we will ever try to allocate. */
+#define BROTLI_MAX_ALLOCABLE_MEMORY (1 << 30)
-// size-checking safe malloc/calloc: verify that the requested size is not too
-// large, or return NULL. You don't need to call these for constructs like
-// malloc(sizeof(foo)), but only if there's font-dependent size involved
-// somewhere (like: malloc(decoded_size * sizeof(*something))). That's why this
-// safe malloc() borrows the signature from calloc(), pointing at the dangerous
-// underlying multiply involved.
+/* size-checking safe malloc/calloc: verify that the requested size is not too
+ large, or return NULL. You don't need to call these for constructs like
+ malloc(sizeof(foo)), but only if there's font-dependent size involved
+ somewhere (like: malloc(decoded_size * sizeof(*something))). That's why this
+ safe malloc() borrows the signature from calloc(), pointing at the dangerous
+ underlying multiply involved.
+*/
void* BrotliSafeMalloc(uint64_t nmemb, size_t size);
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
#endif /* BROTLI_UTILS_UTILS_H_ */
diff --git a/brotli/dec/streams.c b/brotli/dec/streams.c
index 89e030a..ac0f07e 100644
--- a/brotli/dec/streams.c
+++ b/brotli/dec/streams.c
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Functions for streaming input and output.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Functions for streaming input and output.
+*/
#include <string.h>
#ifndef _WIN32
@@ -71,7 +72,7 @@ BrotliOutput BrotliInitMemOutput(uint8_t* buffer, size_t length,
int BrotliStdinInputFunction(void* data, uint8_t* buf, size_t count) {
#ifndef _WIN32
- return read(STDIN_FILENO, buf, count);
+ return (int)read(STDIN_FILENO, buf, count);
#else
return -1;
#endif
@@ -86,7 +87,7 @@ BrotliInput BrotliStdinInput() {
int BrotliStdoutOutputFunction(void* data, const uint8_t* buf, size_t count) {
#ifndef _WIN32
- return write(STDOUT_FILENO, buf, count);
+ return (int)write(STDOUT_FILENO, buf, count);
#else
return -1;
#endif
@@ -112,5 +113,5 @@ BrotliOutput BrotliFileOutput(FILE* f) {
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
diff --git a/brotli/dec/streams.h b/brotli/dec/streams.h
index b055234..1c8ef65 100644
--- a/brotli/dec/streams.h
+++ b/brotli/dec/streams.h
@@ -1,18 +1,19 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Functions for streaming input and output.
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Functions for streaming input and output.
+*/
#ifndef BROTLI_DEC_STREAMS_H_
#define BROTLI_DEC_STREAMS_H_
@@ -24,79 +25,79 @@
extern "C" {
#endif
-// Function pointer type used to read len bytes into buf. Returns the
-// number of bytes read or -1 on error.
+/* Function pointer type used to read len bytes into buf. Returns the */
+/* number of bytes read or -1 on error. */
typedef int (*BrotliInputFunction)(void* data, uint8_t* buf, size_t len);
-// Input callback function with associated data.
+/* Input callback function with associated data. */
typedef struct {
BrotliInputFunction cb_;
void* data_;
} BrotliInput;
-// Reads len bytes into buf, using the in callback.
+/* Reads len bytes into buf, using the in callback. */
static BROTLI_INLINE int BrotliRead(BrotliInput in, uint8_t* buf, size_t len) {
return in.cb_(in.data_, buf, len);
}
-// Function pointer type used to write len bytes into buf. Returns the
-// number of bytes written or -1 on error.
+/* Function pointer type used to write len bytes into buf. Returns the */
+/* number of bytes written or -1 on error. */
typedef int (*BrotliOutputFunction)(void* data, const uint8_t* buf, size_t len);
-// Output callback function with associated data.
+/* Output callback function with associated data. */
typedef struct {
BrotliOutputFunction cb_;
void* data_;
} BrotliOutput;
-// Writes len bytes into buf, using the out callback.
+/* Writes len bytes into buf, using the out callback. */
static BROTLI_INLINE int BrotliWrite(BrotliOutput out,
const uint8_t* buf, size_t len) {
return out.cb_(out.data_, buf, len);
}
-// Memory region with position.
+/* Memory region with position. */
typedef struct {
const uint8_t* buffer;
size_t length;
size_t pos;
} BrotliMemInput;
-// Input callback where *data is a BrotliMemInput struct.
+/* Input callback where *data is a BrotliMemInput struct. */
int BrotliMemInputFunction(void* data, uint8_t* buf, size_t count);
-// Returns an input callback that wraps the given memory region.
+/* Returns an input callback that wraps the given memory region. */
BrotliInput BrotliInitMemInput(const uint8_t* buffer, size_t length,
BrotliMemInput* mem_input);
-// Output buffer with position.
+/* Output buffer with position. */
typedef struct {
uint8_t* buffer;
size_t length;
size_t pos;
} BrotliMemOutput;
-// Output callback where *data is a BrotliMemOutput struct.
+/* Output callback where *data is a BrotliMemOutput struct. */
int BrotliMemOutputFunction(void* data, const uint8_t* buf, size_t count);
-// Returns an output callback that wraps the given memory region.
+/* Returns an output callback that wraps the given memory region. */
BrotliOutput BrotliInitMemOutput(uint8_t* buffer, size_t length,
BrotliMemOutput* mem_output);
-// Input callback that reads from standard input.
+/* Input callback that reads from standard input. */
int BrotliStdinInputFunction(void* data, uint8_t* buf, size_t count);
BrotliInput BrotliStdinInput();
-// Output callback that writes to standard output.
+/* Output callback that writes to standard output. */
int BrotliStdoutOutputFunction(void* data, const uint8_t* buf, size_t count);
BrotliOutput BrotliStdoutOutput();
-// Output callback that writes to a file.
+/* Output callback that writes to a file. */
int BrotliFileOutputFunction(void* data, const uint8_t* buf, size_t count);
BrotliOutput BrotliFileOutput(FILE* f);
#if defined(__cplusplus) || defined(c_plusplus)
-} // extern "C"
+} /* extern "C" */
#endif
-#endif // BROTLI_DEC_STREAMS_H_
+#endif /* BROTLI_DEC_STREAMS_H_ */
diff --git a/brotli/dec/types.h b/brotli/dec/types.h
index 41696e4..bc09f8b 100644
--- a/brotli/dec/types.h
+++ b/brotli/dec/types.h
@@ -1,23 +1,24 @@
-// Copyright 2013 Google Inc. All Rights Reserved.
-//
-// Licensed under the Apache License, Version 2.0 (the "License");
-// you may not use this file except in compliance with the License.
-// You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing, software
-// distributed under the License is distributed on an "AS IS" BASIS,
-// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-// See the License for the specific language governing permissions and
-// limitations under the License.
-//
-// Common types
+/* Copyright 2013 Google Inc. All Rights Reserved.
+
+ Licensed under the Apache License, Version 2.0 (the "License");
+ you may not use this file except in compliance with the License.
+ You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+
+ Common types
+*/
#ifndef BROTLI_DEC_TYPES_H_
#define BROTLI_DEC_TYPES_H_
-#include <stddef.h> // for size_t
+#include <stddef.h> /* for size_t */
#ifndef _MSC_VER
#include <inttypes.h>
@@ -38,4 +39,4 @@ typedef long long int int64_t;
#define BROTLI_INLINE __forceinline
#endif /* _MSC_VER */
-#endif // BROTLI_DEC_TYPES_H_
+#endif /* BROTLI_DEC_TYPES_H_ */
diff --git a/brotli/enc/backward_references.cc b/brotli/enc/backward_references.cc
index 71554fe..0e7f89b 100644
--- a/brotli/enc/backward_references.cc
+++ b/brotli/enc/backward_references.cc
@@ -87,7 +87,7 @@ void CreateBackwardReferences(size_t num_bytes,
// If we are not inserting any symbols, inserting one is more
// expensive than if we were inserting symbols anyways.
if (insert_length < 1) {
- cost_diff_lazy += 1.0;
+ cost_diff_lazy += 0.97;
}
// Add bias to slightly avoid lazy matching.
cost_diff_lazy += 2.0 + delayed_backward_references_in_row * 0.2;
diff --git a/brotli/enc/bit_cost.h b/brotli/enc/bit_cost.h
index 5d6ef0f..c769455 100644
--- a/brotli/enc/bit_cost.h
+++ b/brotli/enc/bit_cost.h
@@ -104,7 +104,7 @@ static inline int HuffmanBitCost(const uint8_t* depth, int length) {
template<int kSize>
double PopulationCost(const Histogram<kSize>& histogram) {
if (histogram.total_count_ == 0) {
- return 11;
+ return 12;
}
int count = 0;
for (int i = 0; i < kSize && count < 5; ++i) {
@@ -113,10 +113,10 @@ double PopulationCost(const Histogram<kSize>& histogram) {
}
}
if (count == 1) {
- return 11;
+ return 12;
}
if (count == 2) {
- return 19 + histogram.total_count_;
+ return 20 + histogram.total_count_;
}
uint8_t depth[kSize] = { 0 };
CreateHuffmanTree(&histogram.data_[0], kSize, 15, depth);
@@ -125,7 +125,9 @@ double PopulationCost(const Histogram<kSize>& histogram) {
bits += histogram.data_[i] * depth[i];
}
if (count == 3) {
- bits += 27;
+ bits += 28;
+ } else if (count == 4) {
+ bits += 37;
} else {
bits += HuffmanBitCost(depth, kSize);
}
diff --git a/brotli/enc/block_splitter.cc b/brotli/enc/block_splitter.cc
index e3d7363..34363c4 100644
--- a/brotli/enc/block_splitter.cc
+++ b/brotli/enc/block_splitter.cc
@@ -82,18 +82,6 @@ void CopyCommandsToByteArray(const std::vector<Command>& cmds,
}
}
-template<int kSize>
-double HistogramAddEval(const Histogram<kSize>& a,
- const Histogram<kSize>& b) {
- int total = a.total_count_ + b.total_count_;
- double retval = total * FastLog2(total);
- for (int i = 0; i < kSize; ++i) {
- int count = a.data_[i] + b.data_[i];
- retval -= count * FastLog2(count);
- }
- return retval;
-}
-
template<typename HistogramType, typename DataType>
void InitialEntropyCodes(const DataType* data, size_t length,
int literals_per_histogram,
@@ -120,28 +108,19 @@ void InitialEntropyCodes(const DataType* data, size_t length,
}
}
-template<typename HistogramType>
-int FindClosest(const HistogramType& sample,
- const std::vector<HistogramType>& vec) {
- double best_distance = 1e99;
- int best_ix = 0;
- for (int i = 0; i < vec.size(); ++i) {
- double distance = HistogramAddEval(sample, vec[i]);
- if (distance < best_distance) {
- best_ix = i;
- best_distance = distance;
- }
- }
- return best_ix;
-}
-
template<typename HistogramType, typename DataType>
void RandomSample(unsigned int* seed,
const DataType* data,
size_t length,
size_t stride,
HistogramType* sample) {
- size_t pos = rand_r(seed) % (length - stride);
+ size_t pos = 0;
+ if (stride >= length) {
+ pos = 0;
+ stride = length;
+ } else {
+ pos = rand_r(seed) % (length - stride + 1);
+ }
sample->Add(data + pos, stride);
}
@@ -149,13 +128,14 @@ template<typename HistogramType, typename DataType>
void RefineEntropyCodes(const DataType* data, size_t length,
size_t stride,
std::vector<HistogramType>* vec) {
- const int iters =
+ int iters =
kIterMulForRefining * length / stride + kMinItersForRefining;
unsigned int seed = 7;
+ iters = ((iters + vec->size() - 1) / vec->size()) * vec->size();
for (int iter = 0; iter < iters; ++iter) {
HistogramType sample;
RandomSample(&seed, data, length, stride, &sample);
- int ix = FindClosest(sample, *vec);
+ int ix = iter % vec->size();
(*vec)[ix].AddHistogram(sample);
}
}
diff --git a/brotli/enc/encode.cc b/brotli/enc/encode.cc
index 7d54dbe..b492421 100644
--- a/brotli/enc/encode.cc
+++ b/brotli/enc/encode.cc
@@ -122,18 +122,30 @@ void StoreHuffmanTreeOfHuffmanTreeToBitMask(
};
// Throw away trailing zeros:
int codes_to_store = kCodeLengthCodes;
- for (; codes_to_store > 3; --codes_to_store) {
+ for (; codes_to_store > 0; --codes_to_store) {
if (code_length_bitdepth[kStorageOrder[codes_to_store - 1]] != 0) {
break;
}
}
- WriteBits(4, codes_to_store - 3, storage_ix, storage);
- const int skip_two_first =
- code_length_bitdepth[kStorageOrder[0]] == 0 &&
- code_length_bitdepth[kStorageOrder[1]] == 0;
- WriteBits(1, skip_two_first, storage_ix, storage);
-
- for (int i = skip_two_first * 2; i < codes_to_store; ++i) {
+ int num_codes = 0;
+ for (int i = 0; i < codes_to_store; ++i) {
+ if (code_length_bitdepth[kStorageOrder[i]] != 0) {
+ ++num_codes;
+ }
+ }
+ if (num_codes == 1) {
+ codes_to_store = kCodeLengthCodes;
+ }
+ int skip_some = 0; // skips none.
+ if (code_length_bitdepth[kStorageOrder[0]] == 0 &&
+ code_length_bitdepth[kStorageOrder[1]] == 0) {
+ skip_some = 2; // skips two.
+ if (code_length_bitdepth[kStorageOrder[2]] == 0) {
+ skip_some = 3; // skips three.
+ }
+ }
+ WriteBits(2, skip_some, storage_ix, storage);
+ for (int i = skip_some; i < codes_to_store; ++i) {
uint8_t len[] = { 2, 4, 3, 2, 2, 4 };
uint8_t bits[] = { 0, 5, 1, 3, 2, 13 };
int v = code_length_bitdepth[kStorageOrder[i]];
@@ -174,7 +186,7 @@ void StoreHuffmanCode(const EntropyCode<kSize>& code, int alphabet_size,
}
if (code.count_ == 0) { // emit minimal tree for empty cases
// bits: small tree marker: 1, count-1: 0, max_bits-sized encoding for 0
- WriteBits(3 + max_bits, 0x01, storage_ix, storage);
+ WriteBits(4 + max_bits, 0x1, storage_ix, storage);
return;
}
if (code.count_ <= 4) {
@@ -194,7 +206,7 @@ void StoreHuffmanCode(const EntropyCode<kSize>& code, int alphabet_size,
}
}
// Small tree marker to encode 1-4 symbols.
- WriteBits(1, 1, storage_ix, storage);
+ WriteBits(2, 1, storage_ix, storage);
WriteBits(2, code.count_ - 1, storage_ix, storage);
for (int i = 0; i < code.count_; ++i) {
WriteBits(max_bits, symbols[i], storage_ix, storage);
@@ -211,8 +223,6 @@ void StoreHuffmanCode(const EntropyCode<kSize>& code, int alphabet_size,
}
return;
}
- WriteBits(1, 0, storage_ix, storage);
-
uint8_t huffman_tree[kSize];
uint8_t huffman_tree_extra_bits[kSize];
int huffman_tree_size = 0;
@@ -229,40 +239,8 @@ void StoreHuffmanCode(const EntropyCode<kSize>& code, int alphabet_size,
EntropyCode<kCodeLengthCodes> huffman_tree_entropy;
BuildEntropyCode(huffman_tree_histogram, 5, kCodeLengthCodes,
&huffman_tree_entropy);
- Histogram<kCodeLengthCodes> trimmed_histogram = huffman_tree_histogram;
- uint8_t* last_code = &huffman_tree[huffman_tree_size - 1];
- while (*last_code == 0 || *last_code >= 17) {
- trimmed_histogram.Remove(*last_code--);
- }
- int trimmed_size = trimmed_histogram.total_count_;
- bool write_length = false;
- if (trimmed_size >= 4 && trimmed_size <= 195 &&
- trimmed_size < huffman_tree_size) {
- EntropyCode<kCodeLengthCodes> trimmed_entropy;
- BuildEntropyCode(trimmed_histogram, 5, kCodeLengthCodes, &trimmed_entropy);
- int huffman_bit_cost = HuffmanTreeBitCost(huffman_tree_histogram,
- huffman_tree_entropy);
- int trimmed_bit_cost = HuffmanTreeBitCost(trimmed_histogram,
- trimmed_entropy);;
- trimmed_bit_cost += (trimmed_size < 68 ? 7 : 8);
- if (trimmed_bit_cost < huffman_bit_cost) {
- write_length = true;
- huffman_tree_size = trimmed_size;
- huffman_tree_entropy = trimmed_entropy;
- }
- }
-
StoreHuffmanTreeOfHuffmanTreeToBitMask(
&huffman_tree_entropy.depth_[0], storage_ix, storage);
- WriteBits(1, write_length, storage_ix, storage);
- if (write_length) {
- WriteBits(1, huffman_tree_size >= 68, storage_ix, storage);
- if (huffman_tree_size < 68) {
- WriteBits(6, huffman_tree_size - 4, storage_ix, storage);
- } else {
- WriteBits(7, huffman_tree_size - 68, storage_ix, storage);
- }
- }
StoreHuffmanTreeToBitMask(&huffman_tree[0], &huffman_tree_extra_bits[0],
huffman_tree_size, huffman_tree_entropy,
storage_ix, storage);
@@ -322,9 +300,13 @@ void ComputeDistanceShortCodes(std::vector<Command>* cmds,
int cur_dist = (*cmds)[i].copy_distance_;
if (cur_dist == 0) break;
int dist_code = cur_dist + 16;
+ int limits[16] = { 0, 4, 10, 11,
+ 6, 6, 11, 11,
+ 11, 11, 11, 11,
+ 12, 12, 12, 12 };
for (int k = 0; k < 16; ++k) {
// Only accept more popular choices.
- if (cur_dist < 11 && ((k >= 2 && k < 4) || k >= 6)) {
+ if (cur_dist < limits[k]) {
// Typically unpopular ranges, don't replace a short distance
// with them.
continue;
diff --git a/brotli/enc/entropy_encode.cc b/brotli/enc/entropy_encode.cc
index ad9f0f5..e4c6b20 100644
--- a/brotli/enc/entropy_encode.cc
+++ b/brotli/enc/entropy_encode.cc
@@ -352,6 +352,12 @@ void WriteHuffmanTree(const uint8_t* depth, const int length,
}
i += reps;
}
+ // Throw away trailing zeros.
+ for (; *huffman_tree_size > 0; --(*huffman_tree_size)) {
+ if (tree[*huffman_tree_size - 1] > 0 && tree[*huffman_tree_size - 1] < 17) {
+ break;
+ }
+ }
}
namespace {
diff --git a/brotli/enc/hash.h b/brotli/enc/hash.h
index b45d71c..cb38e8f 100644
--- a/brotli/enc/hash.h
+++ b/brotli/enc/hash.h
@@ -205,7 +205,9 @@ class HashLongestMatch {
continue;
}
prev_ix &= ring_buffer_mask;
- if (data[cur_ix_masked + best_len] != data[prev_ix + best_len]) {
+ if (cur_ix_masked + best_len > ring_buffer_mask ||
+ prev_ix + best_len > ring_buffer_mask ||
+ data[cur_ix_masked + best_len] != data[prev_ix + best_len]) {
continue;
}
const size_t len =
@@ -277,7 +279,9 @@ class HashLongestMatch {
break;
}
prev_ix &= ring_buffer_mask;
- if (data[cur_ix_masked + best_len] != data[prev_ix + best_len]) {
+ if (cur_ix_masked + best_len > ring_buffer_mask ||
+ prev_ix + best_len > ring_buffer_mask ||
+ data[cur_ix_masked + best_len] != data[prev_ix + best_len]) {
continue;
}
const size_t len =
diff --git a/brotli/enc/literal_cost.cc b/brotli/enc/literal_cost.cc
index 2a388d7..bf05a98 100644
--- a/brotli/enc/literal_cost.cc
+++ b/brotli/enc/literal_cost.cc
@@ -51,7 +51,7 @@ void EstimateBitCostsForLiterals(size_t pos, size_t len, size_t mask,
histo = 1;
}
cost[masked_pos] = log2(static_cast<double>(in_window) / histo);
- cost[masked_pos] += 0.03;
+ cost[masked_pos] += 0.029;
if (cost[masked_pos] < 1.0) {
cost[masked_pos] *= 0.5;
cost[masked_pos] += 0.5;