Receiving invalid UTF-8 TEXT message should fail and close the connection (fix remaining autobahn test: 6.X UTF-8 Handling)

CMakeLists.txt

@@ -69,6 +69,7 @@ set( IXWEBSOCKET_HEADERS
     ixwebsocket/IXSocketFactory.h
     ixwebsocket/IXSocketServer.h
     ixwebsocket/IXUrlParser.h
+    ixwebsocket/IXUtf8Validator.h
     ixwebsocket/IXUserAgent.h
     ixwebsocket/IXWebSocket.h
     ixwebsocket/IXWebSocketCloseConstants.h

DOCKER_VERSION

@@ -1 +1 @@
-5.1.1
+5.1.7

docs/CHANGELOG.md

@@ -1,27 +1,50 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## [5.1.7] - 2019-09-03
+
+- Receiving invalid UTF-8 TEXT message should fail and close the connection (fix remaining autobahn test: 6.X UTF-8 Handling)
+
+## [5.1.6] - 2019-09-03
+
+- Sending invalid UTF-8 TEXT message should fail and close the connection (fix remaining autobahn test: 6.X UTF-8 Handling)
+- Fix failing unittest which was sending binary data in text mode with WebSocket::send to call properly call WebSocket::sendBinary instead.
+- Validate that the reason is proper utf-8. (fix autobahn test 7.5.1)
+- Validate close codes. Autobahn 7.9.*
+
+## [5.1.5] - 2019-09-03
+
+Framentation: data and continuation blocks received out of order (fix autobahn test: 5.9 through 5.20 Fragmentation)
+
+## [5.1.4] - 2019-09-03
+
+Sending invalid UTF-8 TEXT message should fail and close the connection (fix **tons** of autobahn test: 6.X UTF-8 Handling)
+
+## [5.1.3] - 2019-09-03
+
+Message type (TEXT or BINARY) is invalid for received fragmented messages (fix autobahn test: 5.3 through 5.8 Fragmentation)
+
 ## [5.1.2] - 2019-09-02
 
-Ping and Pong messages cannot be fragmented (autobahn test: 5.1 and 5.2 Fragmentation)
+Ping and Pong messages cannot be fragmented (fix autobahn test: 5.1 and 5.2 Fragmentation)
 
 ## [5.1.1] - 2019-09-01
 
-Close connections when reserved bits are used (autobahn test: 3 Reserved Bits)
+Close connections when reserved bits are used (fix autobahn test: 3.X Reserved Bits)
 
 ## [5.1.0] - 2019-08-31
 
-ws autobahn / Add code to test websocket client compliance with the autobahn test-suite
+- ws autobahn / Add code to test websocket client compliance with the autobahn test-suite
-add utf-8 validation code, not hooked up properly yet
+- add utf-8 validation code, not hooked up properly yet
-Ping received with a payload too large (> 125 bytes) trigger a connection closure
+- Ping received with a payload too large (> 125 bytes) trigger a connection closure
-cobra / add tracking about published messages
+- cobra / add tracking about published messages
-cobra / publish returns a message id, that can be used when 
+- cobra / publish returns a message id, that can be used when 
-cobra / new message type in the message received handler when publish/ok is received (can be used to implement an ack system).
+- cobra / new message type in the message received handler when publish/ok is received (can be used to implement an ack system).
 
 ## [5.0.9] - 2019-08-30
 
-User-Agent header is set when not specified.
+- User-Agent header is set when not specified.
-New option to cap the max wait between reconnection attempts. Still default to 10s. (setMaxWaitBetweenReconnectionRetries).
+- New option to cap the max wait between reconnection attempts. Still default to 10s. (setMaxWaitBetweenReconnectionRetries).
 
 ```
 ws connect --max_wait 5000 ws://example.com # will only wait 5 seconds max between reconnection attempts

docs/design.md

@@ -32,7 +32,6 @@ The regression test is running after each commit on travis.
 
 * On Windows TLS is not setup yet to validate certificates.
 * There is no convenient way to embed a ca cert.
-* No utf-8 validation is made when sending TEXT message with sendText()
 * Automatic reconnection works at the TCP socket level, and will detect remote end disconnects. However, if the device/computer network become unreachable (by turning off wifi), it is quite hard to reliably and timely detect it at the socket level using `recv` and `send` error codes. [Here](https://stackoverflow.com/questions/14782143/linux-socket-how-to-detect-disconnected-network-in-a-client-program) is a good discussion on the subject. This behavior is consistent with other runtimes such as node.js. One way to detect a disconnected device with low level C code is to do a name resolution with DNS but this can be expensive. Mobile devices have good and reliable API to do that.
 * The server code is using select to detect incoming data, and creates one OS thread per connection. This is not as scalable as strategies using epoll or kqueue.
 

ixwebsocket/IXUtf8Validator.h

@@ -0,0 +1,167 @@
+/*
+ * The following code is adapted from code originally written by Bjoern
+ * Hoehrmann <bjoern@hoehrmann.de>. See
+ * http://bjoern.hoehrmann.de/utf-8/decoder/dfa/ for details.
+ *
+ * The original license:
+ *
+ * Copyright (c) 2008-2009 Bjoern Hoehrmann <bjoern@hoehrmann.de>
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+*/
+
+/*
+ *  IXUtf8Validator.h
+ *  Author: Benjamin Sergeant
+ *  Copyright (c) 2019 Machine Zone, Inc. All rights reserved.
+ *
+ *  From websocketpp. Tiny modifications made for code style, function names etc...
+ */
+
+#pragma once
+
+#include <cstdint>
+#include <string>
+
+namespace ix
+{
+    /// State that represents a valid utf8 input sequence
+    static unsigned int const utf8_accept = 0;
+    /// State that represents an invalid utf8 input sequence
+    static unsigned int const utf8_reject = 1;
+
+    /// Lookup table for the UTF8 decode state machine
+    static uint8_t const utf8d[] = {
+      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, // 00..1f
+      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, // 20..3f
+      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, // 40..5f
+      0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0, // 60..7f
+      1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9,9, // 80..9f
+      7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7,7, // a0..bf
+      8,8,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2,2, // c0..df
+      0xa,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x3,0x4,0x3,0x3, // e0..ef
+      0xb,0x6,0x6,0x6,0x5,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8,0x8, // f0..ff
+      0x0,0x1,0x2,0x3,0x5,0x8,0x7,0x1,0x1,0x1,0x4,0x6,0x1,0x1,0x1,0x1, // s0..s0
+      1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0,1,1,1,1,1,0,1,0,1,1,1,1,1,1, // s1..s2
+      1,2,1,1,1,1,1,2,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1, // s3..s4
+      1,2,1,1,1,1,1,1,1,2,1,1,1,1,1,1,1,1,1,1,1,1,1,3,1,3,1,1,1,1,1,1, // s5..s6
+      1,3,1,1,1,1,1,3,1,3,1,1,1,1,1,1,1,3,1,1,1,1,1,1,1,1,1,1,1,1,1,1, // s7..s8
+    };
+
+    /// Decode the next byte of a UTF8 sequence
+    /**
+     * @param [out] state The decoder state to advance
+     * @param [out] codep The codepoint to fill in
+     * @param [in] byte The byte to input
+     * @return The ending state of the decode operation
+     */
+    inline uint32_t decodeNextByte(uint32_t * state, uint32_t * codep, uint8_t byte)
+    {
+      uint32_t type = utf8d[byte];
+
+      *codep = (*state != utf8_accept) ?
+        (byte & 0x3fu) | (*codep << 6) :
+        (0xff >> type) & (byte);
+
+      *state = utf8d[256 + *state*16 + type];
+      return *state;
+    }
+
+    /// Provides streaming UTF8 validation functionality
+    class Utf8Validator
+    {
+    public:
+        /// Construct and initialize the validator
+        Utf8Validator() : m_state(utf8_accept),m_codepoint(0) {}
+
+        /// Advance the state of the validator with the next input byte
+        /**
+         * @param byte The byte to advance the validation state with
+         * @return Whether or not the byte resulted in a validation error.
+         */
+        bool consume(uint8_t byte)
+        {
+            if (decodeNextByte(&m_state,&m_codepoint,byte) == utf8_reject)
+            {
+                return false;
+            }
+            return true;
+        }
+
+        /// Advance Validator state with input from an iterator pair
+        /**
+         * @param begin Input iterator to the start of the input range
+         * @param end Input iterator to the end of the input range
+         * @return Whether or not decoding the bytes resulted in a validation error.
+         */
+        template <typename iterator_type>
+        bool decode(iterator_type begin, iterator_type end)
+        {
+            for (iterator_type it = begin; it != end; ++it)
+            {
+                unsigned int result = decodeNextByte(
+                    &m_state,
+                    &m_codepoint,
+                    static_cast<uint8_t>(*it)
+                );
+
+                if (result == utf8_reject)
+                {
+                    return false;
+                }
+            }
+            return true;
+        }
+
+        /// Return whether the input sequence ended on a valid utf8 codepoint
+        /**
+         * @return Whether or not the input sequence ended on a valid codepoint.
+         */
+        bool complete()
+        {
+            return m_state == utf8_accept;
+        }
+
+        /// Reset the Validator to decode another message
+        void reset()
+        {
+            m_state = utf8_accept;
+            m_codepoint = 0;
+        }
+    private:
+        uint32_t    m_state;
+        uint32_t    m_codepoint;
+    };
+
+    /// Validate a UTF8 string
+    /**
+     * convenience function that creates a Validator, validates a complete string
+     * and returns the result.
+     */
+    inline bool validateUtf8(std::string const & s)
+    {
+        Utf8Validator v;
+        if (!v.decode(s.begin(),s.end()))
+        {
+            return false;
+        }
+        return v.complete();
+    }
+
+} // namespace ix

ixwebsocket/IXWebSocket.cpp

@@ -8,66 +8,12 @@
 #include "IXSetThreadName.h"
 #include "IXWebSocketHandshake.h"
 #include "IXExponentialBackoff.h"
+#include "IXUtf8Validator.h"
 
 #include <cmath>
 #include <cassert>
+#include <iostream>
 
-namespace
-{
-    //
-    // Stolen from here http://www.zedwood.com/article/cpp-is-valid-utf8-string-function
-    // There doesn't seem to be anything in the C++ library so far to do that.
-    // The closest thing is code for converting from utf-8 to utf-16 or utf-32 but 
-    // that isn't working well for some broken input strings.
-    //
-    bool isValidUtf8(const std::string& str)
-    {
-        size_t i = 0;
-        size_t ix = str.length();
-        int c, n, j;
-        
-        for (; i < ix; i++)
-        {
-            c = (unsigned char) str[i];
-            //if (c==0x09 || c==0x0a || c==0x0d || (0x20 <= c && c <= 0x7e) ) n = 0; // is_printable_ascii
-            if (0x00 <= c && c <= 0x7f)
-            {
-                n = 0; // 0bbbbbbb
-            }
-            else if ((c & 0xE0) == 0xC0)
-            {
-                n = 1; // 110bbbbb
-            }
-            else if ( c==0xed && i<(ix-1) && ((unsigned char)str[i+1] & 0xa0)==0xa0)
-            {
-                return false; //U+d800 to U+dfff
-            }
-            else if ((c & 0xF0) == 0xE0)
-            {
-                n = 2; // 1110bbbb
-            }
-            else if ((c & 0xF8) == 0xF0)
-            {
-                n = 3; // 11110bbb
-            }
-            //else if (($c & 0xFC) == 0xF8) n=4; // 111110bb //byte 5, unnecessary in 4 byte UTF-8
-            //else if (($c & 0xFE) == 0xFC) n=5; // 1111110b //byte 6, unnecessary in 4 byte UTF-8
-            else
-            {
-                return false;
-            }
-
-            for (j=0; j<n && i<ix; j++)
-            {   // n bytes matching 10bbbbbb follow ?
-                if ((++i == ix) || (( (unsigned char)str[i] & 0xC0) != 0x80))
-                {
-                    return false;
-                }
-            }
-        }
-        return true;
-    }
-}
 
 namespace ix
 {
@@ -447,9 +393,7 @@ namespace ix
                                       bool binary,
                                       const OnProgressCallback& onProgressCallback)
     {
-        return sendMessage(data,
+        return (binary) ? sendBinary(data, onProgressCallback) : sendText(data, onProgressCallback);
-                           (binary) ? SendMessageKind::Binary: SendMessageKind::Text,
-                           onProgressCallback);
     }
 
     WebSocketSendInfo WebSocket::sendBinary(const std::string& text,
@@ -461,9 +405,10 @@ namespace ix
     WebSocketSendInfo WebSocket::sendText(const std::string& text,
                                           const OnProgressCallback& onProgressCallback)
     {
-        if (!isValidUtf8(text))
+        if (!validateUtf8(text))
         {
-            stop();
+            close(WebSocketCloseConstants::kInvalidFramePayloadData,
+                  WebSocketCloseConstants::kInvalidFramePayloadDataMessage);
             return false;
         }
         return sendMessage(text, SendMessageKind::Text, onProgressCallback);

ixwebsocket/IXWebSocketCloseConstants.cpp

@@ -11,6 +11,7 @@ namespace ix
     const uint16_t WebSocketCloseConstants::kNormalClosureCode(1000);
     const uint16_t WebSocketCloseConstants::kInternalErrorCode(1011);
     const uint16_t WebSocketCloseConstants::kAbnormalCloseCode(1006);
+    const uint16_t WebSocketCloseConstants::kInvalidFramePayloadData(1007);
     const uint16_t WebSocketCloseConstants::kProtocolErrorCode(1002);
     const uint16_t WebSocketCloseConstants::kNoStatusCodeErrorCode(1005);
 
@@ -23,4 +24,8 @@ namespace ix
     const std::string WebSocketCloseConstants::kProtocolErrorReservedBitUsed("Reserved bit used");
     const std::string WebSocketCloseConstants::kProtocolErrorPingPayloadOversized("Ping reason control frame with payload length > 125 octets");
     const std::string WebSocketCloseConstants::kProtocolErrorCodeControlMessageFragmented("Control message fragmented");
+    const std::string WebSocketCloseConstants::kProtocolErrorCodeDataOpcodeOutOfSequence("Fragmentation: data message out of sequence");
+    const std::string WebSocketCloseConstants::kProtocolErrorCodeContinuationOpCodeOutOfSequence("Fragmentation: continuation opcode out of sequence");
+    const std::string WebSocketCloseConstants::kInvalidFramePayloadDataMessage("Invalid frame payload data");
+    const std::string WebSocketCloseConstants::kInvalidCloseCodeMessage("Invalid close code");
 }

ixwebsocket/IXWebSocketCloseConstants.h

@@ -18,6 +18,7 @@ namespace ix
         static const uint16_t kAbnormalCloseCode;
         static const uint16_t kProtocolErrorCode;
         static const uint16_t kNoStatusCodeErrorCode;
+        static const uint16_t kInvalidFramePayloadData;
 
         static const std::string kNormalClosureMessage;
         static const std::string kInternalErrorMessage;
@@ -28,5 +29,9 @@ namespace ix
         static const std::string kProtocolErrorReservedBitUsed;
         static const std::string kProtocolErrorPingPayloadOversized;
         static const std::string kProtocolErrorCodeControlMessageFragmented;
+        static const std::string kProtocolErrorCodeDataOpcodeOutOfSequence;
+        static const std::string kProtocolErrorCodeContinuationOpCodeOutOfSequence;
+        static const std::string kInvalidFramePayloadDataMessage;
+        static const std::string kInvalidCloseCodeMessage;
     };
 } // namespace ix

ixwebsocket/IXWebSocketTransport.cpp

@@ -37,6 +37,7 @@
 #include "IXWebSocketHttpHeaders.h"
 #include "IXUrlParser.h"
 #include "IXSocketFactory.h"
+#include "IXUtf8Validator.h"
 
 #include <string.h>
 #include <stdlib.h>
@@ -551,33 +552,51 @@ namespace ix
                 || ws.opcode == wsheader_type::PONG
                 || ws.opcode == wsheader_type::CLOSE
             )){
-                // Cntrol messages should not be fragmented
+                // Control messages should not be fragmented
                 close(WebSocketCloseConstants::kProtocolErrorCode,
                       WebSocketCloseConstants::kProtocolErrorCodeControlMessageFragmented);
                 return;
             }
 
+            unmaskReceiveBuffer(ws);
+            std::string frameData(_rxbuf.begin()+ws.header_size,
+                                  _rxbuf.begin()+ws.header_size+(size_t) ws.N);
+
             // We got a whole message, now do something with it:
             if (
                    ws.opcode == wsheader_type::TEXT_FRAME
                 || ws.opcode == wsheader_type::BINARY_FRAME
                 || ws.opcode == wsheader_type::CONTINUATION
             ) {
-                unmaskReceiveBuffer(ws);
 
-                MessageKind messageKind =
+                if (ws.opcode != wsheader_type::CONTINUATION)
-                    (ws.opcode == wsheader_type::TEXT_FRAME)
+                {
-                    ? MessageKind::MSG_TEXT
+                    _fragmentedMessageKind =
-                    : MessageKind::MSG_BINARY;
+                        (ws.opcode == wsheader_type::TEXT_FRAME)
+                        ? MessageKind::MSG_TEXT
+                        : MessageKind::MSG_BINARY;
+
+                    // Continuation message needs to follow a non-fin TEXT or BINARY message
+                    if (!_chunks.empty())
+                    {
+                        close(WebSocketCloseConstants::kProtocolErrorCode,
+                              WebSocketCloseConstants::kProtocolErrorCodeDataOpcodeOutOfSequence);
+                    }
+                }
+                else if (_chunks.empty())
+                {
+                    // Continuation message need to follow a non-fin TEXT or BINARY message
+                    close(WebSocketCloseConstants::kProtocolErrorCode,
+                          WebSocketCloseConstants::kProtocolErrorCodeContinuationOpCodeOutOfSequence);
+                }
 
                 //
                 // Usual case. Small unfragmented messages
                 //
                 if (ws.fin && _chunks.empty())
                 {
-                    emitMessage(messageKind,
+                    emitMessage(_fragmentedMessageKind,
-                                std::string(_rxbuf.begin()+ws.header_size,
+                                frameData,
-                                            _rxbuf.begin()+ws.header_size+(size_t) ws.N),
                                 ws,
                                 onMessageCallback);
                 }
@@ -590,12 +609,12 @@ namespace ix
                     // the internal buffer which is slow and can let the internal OS
                     // receive buffer fill out.
                     //
-                    _chunks.emplace_back(
+                    _chunks.emplace_back(frameData);
-                        std::vector<uint8_t>(_rxbuf.begin()+ws.header_size,
+
-                                             _rxbuf.begin()+ws.header_size+(size_t)ws.N));
                     if (ws.fin)
                     {
-                        emitMessage(messageKind, getMergedChunks(), ws, onMessageCallback);
+                        emitMessage(_fragmentedMessageKind, getMergedChunks(),
+                                    ws, onMessageCallback);
                         _chunks.clear();
                     }
                     else
@@ -606,13 +625,8 @@ namespace ix
             }
             else if (ws.opcode == wsheader_type::PING)
             {
-                unmaskReceiveBuffer(ws);
-
-                std::string pingData(_rxbuf.begin()+ws.header_size,
-                                     _rxbuf.begin()+ws.header_size + (size_t) ws.N);
-
                 // too large
-                if (pingData.size() > 125)
+                if (frameData.size() > 125)
                 {
                     // Unexpected frame type
                     close(WebSocketCloseConstants::kProtocolErrorCode,
@@ -624,29 +638,23 @@ namespace ix
                 {
                     // Reply back right away
                     bool compress = false;
-                    sendData(wsheader_type::PONG, pingData, compress);
+                    sendData(wsheader_type::PONG, frameData, compress);
                 }
 
-                emitMessage(MessageKind::PING, pingData, ws, onMessageCallback);
+                emitMessage(MessageKind::PING, frameData, ws, onMessageCallback);
             }
             else if (ws.opcode == wsheader_type::PONG)
             {
-                unmaskReceiveBuffer(ws);
-                std::string pongData(_rxbuf.begin()+ws.header_size,
-                                     _rxbuf.begin()+ws.header_size + (size_t) ws.N);
-
                 std::lock_guard<std::mutex> lck(_lastReceivePongTimePointMutex);
                 _lastReceivePongTimePoint = std::chrono::steady_clock::now();
 
-                emitMessage(MessageKind::PONG, pongData, ws, onMessageCallback);
+                emitMessage(MessageKind::PONG, frameData, ws, onMessageCallback);
             }
             else if (ws.opcode == wsheader_type::CLOSE)
             {
                 std::string reason;
                 uint16_t code = 0;
 
-                unmaskReceiveBuffer(ws);
-
                 if (ws.N >= 2)
                 {
                     // Extract the close code first, available as the first 2 bytes
@@ -656,8 +664,28 @@ namespace ix
                     // Get the reason.
                     if (ws.N > 2)
                     {
-                        reason.assign(_rxbuf.begin()+ws.header_size + 2,
+                        reason = frameData.substr(2, frameData.size());
-                                      _rxbuf.begin()+ws.header_size + (size_t) ws.N);
+                    }
+
+                    // Validate that the reason is proper utf-8. Autobahn 7.5.1
+                    if (!validateUtf8(reason))
+                    {
+                        code = WebSocketCloseConstants::kInvalidFramePayloadData;
+                        reason = WebSocketCloseConstants::kInvalidFramePayloadDataMessage;
+                    }
+
+                    // Validate close codes. Autobahn 7.9.*
+                    // 1014, 1015 are debattable. The firefox MSDN has a description for them
+                    if (code < 1000 || code == 1004 || code == 1006 ||
+                        (code > 1013 && code < 3000))
+                    {
+                        // build up an error message containing the bad error code
+                        std::stringstream ss;
+                        ss << WebSocketCloseConstants::kInvalidCloseCodeMessage
+                           << ": " << code;
+                        reason = ss.str();
+
+                        code = WebSocketCloseConstants::kProtocolErrorCode;
                     }
                 }
                 else
@@ -743,8 +771,7 @@ namespace ix
 
         for (auto&& chunk : _chunks)
         {
-            std::string str(chunk.begin(), chunk.end());
+            msg += chunk;
-            msg += str;
         }
 
         return msg;
@@ -762,11 +789,28 @@ namespace ix
         {
             std::string decompressedMessage;
             bool success = _perMessageDeflate.decompress(message, decompressedMessage);
-            onMessageCallback(decompressedMessage, wireSize, !success, messageKind);
+
+            if (messageKind == MessageKind::MSG_TEXT && !validateUtf8(decompressedMessage))
+            {
+                close(WebSocketCloseConstants::kInvalidFramePayloadData,
+                      WebSocketCloseConstants::kInvalidFramePayloadDataMessage);
+            }
+            else
+            {
+                onMessageCallback(decompressedMessage, wireSize, !success, messageKind);
+            }
         }
         else
         {
-            onMessageCallback(message, wireSize, false, messageKind);
+            if (messageKind == MessageKind::MSG_TEXT && !validateUtf8(message))
+            {
+                close(WebSocketCloseConstants::kInvalidFramePayloadData,
+                      WebSocketCloseConstants::kInvalidFramePayloadDataMessage);
+            }
+            else
+            {
+                onMessageCallback(message, wireSize, false, messageKind);
+            }
         }
     }
 

ixwebsocket/IXWebSocketTransport.h

@@ -149,7 +149,12 @@ namespace ix
         // messages (tested messages up to 700M) and we cannot put them in a single
         // buffer that is resized, as this operation can be slow when a buffer has its
         // size increased 2 fold, while appending to a list has a fixed cost.
-        std::list<std::vector<uint8_t>> _chunks;
+        std::list<std::string> _chunks;
+
+        // Record the message kind (will be TEXT or BINARY) for a fragmented
+        // message, present in the first chunk, since the final chunk will be a
+        // CONTINUATION opcode and doesn't tell the full message kind
+        MessageKind _fragmentedMessageKind;
 
         // Fragments are 32K long
         static constexpr size_t kChunkSize = 1 << 15;

ixwebsocket/IXWebSocketVersion.h

@@ -6,4 +6,4 @@
 
 #pragma once
 
-#define IX_WEBSOCKET_VERSION "5.1.2"
+#define IX_WEBSOCKET_VERSION "5.1.7"

test/cmd_websocket_chat.cpp

@@ -206,7 +206,7 @@ namespace
 
     void WebSocketChat::sendMessage(const std::string& text)
     {
-        _webSocket.send(encodeMessage(text));
+        _webSocket.sendBinary(encodeMessage(text));
     }
 
     bool startServer(ix::WebSocketServer& server)
@@ -239,7 +239,7 @@ namespace
                             {
                                 if (client != webSocket)
                                 {
-                                    client->send(msg->str);
+                                    client->sendBinary(msg->str);
                                 }
                             }
                         }