(http client) use std::unordered_map instead of std::map for HttpParameters and HttpFormDataParameters class aliases

* 1) IXWebSocketTransport: BUG: int type has no warranty of number of bits. It depends on compiler and architecture. In my system (64 bit) is 32 bit.
1 << 63 is bad idea in this case because the final number is 0 by overflow.
The symptom observed is that the server can't receive messages.

2) IXSocketFactory: Compilation Warning: Variable not in use.

* Better aproach suggested by Benjamin.

DOCKER_VERSION

@@ -1 +1 @@
-7.3.1
+7.4.3

Dockerfile

@@ -19,6 +19,7 @@ RUN [ "make", "ws_install" ]
 FROM alpine as runtime
 
 RUN apk add --no-cache libstdc++
+RUN apk add --no-cache strace
 
 RUN addgroup -S app && adduser -S -G app app 
 COPY --chown=app:app --from=build /usr/local/bin/ws /usr/local/bin/ws

README.md

@@ -6,7 +6,7 @@ IXWebSocket is a C++ library for WebSocket client and server development. It has
 
 It is been used on big mobile video game titles sending and receiving tons of messages since 2017 (iOS and Android). It was tested on macOS, iOS, Linux, Android, Windows and FreeBSD. Two important design goals are simplicity and correctness.
 
-```
+```cpp
 # Required on Windows
 ix::initNetSystem();
 

docker-compose.yml

@@ -1,43 +1,67 @@
 version: "3"
 services:
-  snake:
-    image: bsergean/ws:build
-    entrypoint: ws snake --port 8765 --host 0.0.0.0 --redis_hosts redis1
+  # snake:
+  #   image: bsergean/ws:build
+  #   entrypoint: ws snake --port 8767 --host 0.0.0.0 --redis_hosts redis1
+  #   ports:
+  #     - "8767:8767"
+  #   networks:
+  #     - ws-net
+  #   depends_on:
+  #     - redis1
+
+  # proxy:
+  #   image: bsergean/ws:build
+  #   entrypoint: strace ws proxy_server --remote_host 'wss://cobra.addsrv.com' --host 0.0.0.0 --port 8765 -v
+  #   ports:
+  #     - "8765:8765"
+  #   networks:
+  #     - ws-net
+
+  pyproxy:
+    image: bsergean/ws_proxy:build
+    entrypoint: /usr/bin/ws_proxy.py --remote_url 'wss://cobra.addsrv.com' --host 0.0.0.0 --port 8765
     ports:
       - "8765:8765"
     networks:
       - ws-net
-    depends_on:
-      - redis1
 
-  ws:
-    security_opt:
-    - seccomp:unconfined
-    cap_add:
-    - SYS_PTRACE
-    stdin_open: true
-    tty: true
-    image: bsergean/ws:build
-    entrypoint: bash
-    networks:
-      - ws-net
-    depends_on:
-      - redis1
+        #  ws:
+        #    security_opt:
+        #    - seccomp:unconfined
+        #    cap_add:
+        #    - SYS_PTRACE
+        #    stdin_open: true
+        #    tty: true
+        #    image: bsergean/ws:build
+        #    entrypoint: sh
+        #    networks:
+        #      - ws-net
+        #    depends_on:
+        #      - redis1
+        #
+        #  redis1:
+        #    image: redis:alpine
+        #    networks:
+        #      - ws-net
+        #
+        #  statsd:
+        #    image: jaconel/statsd
+        #    ports:
+        #      - "8125:8125"
+        #    environment:
+        #      - STATSD_DUMP_MSG=true
+        #      - GRAPHITE_HOST=127.0.0.1
+        #    networks:
+        #      - ws-net
 
-  redis1:
-    image: redis:alpine
-    networks:
-      - ws-net
-
-  statsd:
-    image: jaconel/statsd
-    ports:
-      - "8125:8125"
-    environment:
-      - STATSD_DUMP_MSG=true
-      - GRAPHITE_HOST=127.0.0.1
-    networks:
-      - ws-net
+#  compile:
+#    image: alpine
+#    entrypoint: sh
+#    stdin_open: true
+#    tty: true
+#    volumes:
+#      - /Users/bsergeant/src/foss:/home/bsergean/src/foss
 
 networks:
   ws-net:

docs/CHANGELOG.md

@@ -1,6 +1,35 @@
 # Changelog
 All notable changes to this project will be documented in this file.
 
+## [7.4.3] - 2019-12-03
+
+- (http client) use std::unordered_map instead of std::map for HttpParameters and HttpFormDataParameters class aliases
+
+## [7.4.2] - 2019-12-02
+
+- (client) internal IXDNSLookup class requires a valid cancellation request function callback to be passed in
+
+## [7.4.1] - 2019-12-02
+
+- (client) fix an overflow in the exponential back off code
+
+## [7.4.0] - 2019-11-25
+
+- (http client) Add support for multipart HTTP POST upload
+- (ixsentry) Add support for uploading a minidump to sentry
+
+## [7.3.5] - 2019-11-20
+
+- On Darwin SSL, add ability to skip peer verification.
+
+## [7.3.4] - 2019-11-20
+
+- 32-bits compile fix, courtesy of @fcojavmc
+
+## [7.3.1] - 2019-11-16
+
+- ws proxy_server / remote server close not forwarded to the client
+
 ## [7.3.0] - 2019-11-15
 
 - New ws command: `ws proxy_server`.

docs/cobra.md

@@ -0,0 +1,81 @@
+## General
+
+[cobra](https://github.com/machinezone/cobra) is a real time messaging server. The `ws` utility can run a cobra server (named snake), and has client to publish and subscribe to a cobra server.
+
+Bring up 3 terminals and run a server, a publisher and a subscriber in each one. As you publish data you should see it being received by the subscriber. You can run `redis-cli MONITOR` too to see how redis is being used.
+
+### Server
+
+You will need to have a redis server running locally. To run the server:
+
+```
+$ cd <ixwebsocket-top-level-folder>/ixsnake/ixsnake
+$ ws snake
+{
+  "apps": {
+    "FC2F10139A2BAc53BB72D9db967b024f": {
+      "roles": {
+        "_sub": {
+          "secret": "66B1dA3ED5fA074EB5AE84Dd8CE3b5ba"
+        },
+        "_pub": {
+          "secret": "1c04DB8fFe76A4EeFE3E318C72d771db"
+        }
+      }
+    }
+  }
+}
+
+redis host: 127.0.0.1
+redis password:
+redis port: 6379
+```
+
+### Publisher
+
+```
+$ cd <ixwebsocket-top-level-folder>/ws
+$ ws cobra_publish --appkey FC2F10139A2BAc53BB72D9db967b024f --endpoint ws://127.0.0.1:8008 --rolename _pub --rolesecret 1c04DB8fFe76A4EeFE3E318C72d771db test_channel cobraMetricsSample.json
+[2019-11-27 09:06:12.980] [info] Publisher connected
+[2019-11-27 09:06:12.980] [info] Connection: Upgrade
+[2019-11-27 09:06:12.980] [info] Sec-WebSocket-Accept: zTtQKMKbvwjdivURplYXwCVUCWM=
+[2019-11-27 09:06:12.980] [info] Sec-WebSocket-Extensions: permessage-deflate; server_max_window_bits=15; client_max_window_bits=15
+[2019-11-27 09:06:12.980] [info] Server: ixwebsocket/7.4.0 macos ssl/DarwinSSL zlib 1.2.11
+[2019-11-27 09:06:12.980] [info] Upgrade: websocket
+[2019-11-27 09:06:12.982] [info] Publisher authenticated
+[2019-11-27 09:06:12.982] [info] Published msg 3
+[2019-11-27 09:06:12.982] [info] Published message id 3 acked
+```
+
+### Subscriber
+
+```
+$ ws cobra_subscribe --appkey FC2F10139A2BAc53BB72D9db967b024f --endpoint ws://127.0.0.1:8008 --rolename _pub --rolesecret 1c04DB8fFe76A4EeFE3E318C72d771db test_channel
+#messages 0 msg/s 0
+[2019-11-27 09:07:39.341] [info] Subscriber connected
+[2019-11-27 09:07:39.341] [info] Connection: Upgrade
+[2019-11-27 09:07:39.341] [info] Sec-WebSocket-Accept: 9vkQWofz49qMCUlTSptCCwHWm+Q=
+[2019-11-27 09:07:39.341] [info] Sec-WebSocket-Extensions: permessage-deflate; server_max_window_bits=15; client_max_window_bits=15
+[2019-11-27 09:07:39.341] [info] Server: ixwebsocket/7.4.0 macos ssl/DarwinSSL zlib 1.2.11
+[2019-11-27 09:07:39.341] [info] Upgrade: websocket
+[2019-11-27 09:07:39.342] [info] Subscriber authenticated
+[2019-11-27 09:07:39.345] [info] Subscriber: subscribed to channel test_channel
+#messages 0 msg/s 0
+#messages 0 msg/s 0
+#messages 0 msg/s 0
+{"baz":123,"foo":"bar"}
+
+#messages 1 msg/s 1
+#messages 1 msg/s 0
+#messages 1 msg/s 0
+{"baz":123,"foo":"bar"}
+
+{"baz":123,"foo":"bar"}
+
+#messages 3 msg/s 2
+#messages 3 msg/s 0
+{"baz":123,"foo":"bar"}
+
+#messages 4 msg/s 1
+^C
+```

docs/design.md

@@ -30,7 +30,7 @@ The regression test is running after each commit on travis.
 
 ## Limitations
 
-* On Windows TLS is not setup yet to validate certificates.
+* On Windows and Android certificate validation needs to be setup so that SocketTLSOptions.caFile point to a pem file, such as the one distributed by Firefox. Unless that setup is done connecting to a wss endpoint will display an error. On Windows with mbedtls the message will contain `error in handshake : X509 - Certificate verification failed, e.g. CRL, CA or signature check failed`.
 * There is no convenient way to embed a ca cert.
 * Automatic reconnection works at the TCP socket level, and will detect remote end disconnects. However, if the device/computer network become unreachable (by turning off wifi), it is quite hard to reliably and timely detect it at the socket level using `recv` and `send` error codes. [Here](https://stackoverflow.com/questions/14782143/linux-socket-how-to-detect-disconnected-network-in-a-client-program) is a good discussion on the subject. This behavior is consistent with other runtimes such as node.js. One way to detect a disconnected device with low level C code is to do a name resolution with DNS but this can be expensive. Mobile devices have good and reliable API to do that.
 * The server code is using select to detect incoming data, and creates one OS thread per connection. This is not as scalable as strategies using epoll or kqueue.

ixsentry/CMakeLists.txt

@@ -19,7 +19,7 @@ add_library(ixsentry STATIC
 set(IXSENTRY_INCLUDE_DIRS
     .
     ..
-    ../third_party
-    ../third_party/spdlog/include)
+    ../ixcore
+    ../third_party)
 
 target_include_directories( ixsentry PUBLIC ${IXSENTRY_INCLUDE_DIRS} )

ixsentry/ixsentry/IXSentryClient.cpp

@@ -8,8 +8,9 @@
 
 #include <chrono>
 #include <iostream>
+#include <fstream>
 #include <ixwebsocket/IXWebSocketHttpHeaders.h>
-#include <spdlog/spdlog.h>
+#include <ixcore/utils/IXCoreLogger.h>
 
 
 namespace ix
@@ -18,6 +19,7 @@ namespace ix
         : _dsn(dsn)
         , _validDsn(false)
         , _luaFrameRegex("\t([^/]+):([0-9]+): in function ['<]([^/]+)['>]")
+        , _httpClient(std::make_shared<HttpClient>(true))
     {
         const std::regex dsnRegex("(http[s]?)://([^:]+):([^@]+)@([^/]+)/([0-9]+)");
         std::smatch group;
@@ -169,39 +171,64 @@ namespace ix
 
     std::pair<HttpResponsePtr, std::string> SentryClient::send(const Json::Value& msg, bool verbose)
     {
-        auto args = _httpClient.createRequest();
+        auto args = _httpClient->createRequest();
         args->extraHeaders["X-Sentry-Auth"] = SentryClient::computeAuthHeader();
         args->connectTimeout = 60;
         args->transferTimeout = 5 * 60;
         args->followRedirects = true;
         args->verbose = verbose;
-        args->logger = [](const std::string& msg) { spdlog::info("request logger: {}", msg); };
+        args->logger = [](const std::string& msg) { ix::IXCoreLogger::Log(msg.c_str()); };
 
         std::string body = computePayload(msg);
-        HttpResponsePtr response = _httpClient.post(_url, body, args);
-
-        if (verbose)
-        {
-            for (auto it : response->headers)
-            {
-                spdlog::info("{}: {}", it.first, it.second);
-            }
-
-            spdlog::info("Upload size: {}", response->uploadSize);
-            spdlog::info("Download size: {}", response->downloadSize);
-
-            spdlog::info("Status: {}", response->statusCode);
-            if (response->errorCode != HttpErrorCode::Ok)
-            {
-                spdlog::info("error message: {}", response->errorMsg);
-            }
-
-            if (response->headers["Content-Type"] != "application/octet-stream")
-            {
-                spdlog::info("payload: {}", response->payload);
-            }
-        }
+        HttpResponsePtr response = _httpClient->post(_url, body, args);
 
         return std::make_pair(response, body);
     }
+
+    // https://sentry.io/api/12345/minidump?sentry_key=abcdefgh");
+    std::string SentryClient::computeUrl(const std::string& project, const std::string& key)
+    {
+        std::stringstream ss;
+        ss << "https://sentry.io/api/"
+           << project
+           << "/minidump?sentry_key="
+           << key;
+
+        return ss.str();
+    }
+
+    //
+    // curl -v -X POST -F upload_file_minidump=@ws/crash.dmp 'https://sentry.io/api/123456/minidump?sentry_key=12344567890'
+    //
+    void SentryClient::uploadMinidump(
+        const std::string& sentryMetadata,
+        const std::string& minidumpBytes,
+        const std::string& project,
+        const std::string& key,
+        bool verbose,
+        const OnResponseCallback& onResponseCallback)
+    {
+        std::string multipartBoundary = _httpClient->generateMultipartBoundary();
+
+        auto args = _httpClient->createRequest();
+        args->verb = HttpClient::kPost;
+        args->connectTimeout = 60;
+        args->transferTimeout = 5 * 60;
+        args->followRedirects = true;
+        args->verbose = verbose;
+        args->multipartBoundary = multipartBoundary;
+        args->logger = [](const std::string& msg) { ix::IXCoreLogger::Log(msg.c_str()); };
+
+        HttpFormDataParameters httpFormDataParameters;
+        httpFormDataParameters["upload_file_minidump"] = minidumpBytes;
+
+        HttpParameters httpParameters;
+        httpParameters["sentry"] = sentryMetadata;
+
+        args->url = computeUrl(project, key);
+        args->body = _httpClient->serializeHttpFormDataParameters(multipartBoundary, httpFormDataParameters, httpParameters);
+
+
+        _httpClient->performRequest(args, onResponseCallback);
+    }
 } // namespace ix

ixsentry/ixsentry/IXSentryClient.h

@@ -10,6 +10,7 @@
 #include <ixwebsocket/IXHttpClient.h>
 #include <jsoncpp/json/json.h>
 #include <regex>
+#include <memory>
 
 namespace ix
 {
@@ -23,12 +24,24 @@ namespace ix
 
         Json::Value parseLuaStackTrace(const std::string& stack);
 
+        void uploadMinidump(
+            const std::string& sentryMetadata,
+            const std::string& minidumpBytes,
+            const std::string& project,
+            const std::string& key,
+            bool verbose,
+            const OnResponseCallback& onResponseCallback);
+
     private:
         int64_t getTimestamp();
         std::string computeAuthHeader();
         std::string getIso8601();
         std::string computePayload(const Json::Value& msg);
 
+        std::string computeUrl(const std::string& project, const std::string& key);
+
+        void displayReponse(HttpResponsePtr response);
+
         std::string _dsn;
         bool _validDsn;
         std::string _url;
@@ -41,7 +54,7 @@ namespace ix
 
         std::regex _luaFrameRegex;
 
-        HttpClient _httpClient;
+        std::shared_ptr<HttpClient> _httpClient;
     };
 
 } // namespace ix

ixsnake/ixsnake/IXRedisClient.cpp

@@ -29,8 +29,13 @@ namespace ix
             return false;
         }
 
+        CancellationRequest cancellationRequest = []() -> bool
+        {
+            return false;
+        };
+
         std::string errMsg;
-        return _socket->connect(hostname, port, errMsg, nullptr);
+        return _socket->connect(hostname, port, errMsg, cancellationRequest);
     }
 
     void RedisClient::stop()

ixwebsocket/IXDNSLookup.cpp

@@ -61,7 +61,7 @@ namespace ix
         errMsg = "no error";
 
         // Maybe a cancellation request got in before the background thread terminated ?
-        if (isCancellationRequested && isCancellationRequested())
+        if (isCancellationRequested())
         {
             errMsg = "cancellation requested";
             return nullptr;
@@ -107,7 +107,7 @@ namespace ix
             std::this_thread::sleep_for(std::chrono::milliseconds(_wait));
 
             // Were we cancelled ?
-            if (isCancellationRequested && isCancellationRequested())
+            if (isCancellationRequested())
             {
                 errMsg = "cancellation requested";
                 return nullptr;
@@ -115,7 +115,7 @@ namespace ix
         }
 
         // Maybe a cancellation request got in before the bg terminated ?
-        if (isCancellationRequested && isCancellationRequested())
+        if (isCancellationRequested())
         {
             errMsg = "cancellation requested";
             return nullptr;

ixwebsocket/IXExponentialBackoff.cpp

@@ -13,7 +13,7 @@ namespace ix
     uint32_t calculateRetryWaitMilliseconds(uint32_t retry_count,
                                             uint32_t maxWaitBetweenReconnectionRetries)
     {
-        uint32_t wait_time = std::pow(2, retry_count) * 100;
+        uint32_t wait_time = (retry_count < 26) ? (std::pow(2, retry_count) * 100) : 0;
 
         if (wait_time > maxWaitBetweenReconnectionRetries || wait_time == 0)
         {

ixwebsocket/IXHttp.h

@@ -9,6 +9,7 @@
 #include "IXProgressCallback.h"
 #include "IXWebSocketHttpHeaders.h"
 #include <tuple>
+#include <unordered_map>
 
 namespace ix
 {
@@ -65,7 +66,8 @@ namespace ix
     };
 
     using HttpResponsePtr = std::shared_ptr<HttpResponse>;
-    using HttpParameters = std::map<std::string, std::string>;
+    using HttpParameters = std::unordered_map<std::string, std::string>;
+    using HttpFormDataParameters = std::unordered_map<std::string, std::string>;
     using Logger = std::function<void(const std::string&)>;
     using OnResponseCallback = std::function<void(const HttpResponsePtr&)>;
 
@@ -75,6 +77,7 @@ namespace ix
         std::string verb;
         WebSocketHttpHeaders extraHeaders;
         std::string body;
+        std::string multipartBoundary;
         int connectTimeout;
         int transferTimeout;
         bool followRedirects;

ixwebsocket/IXHttpClient.cpp

@@ -13,6 +13,7 @@
 #include <assert.h>
 #include <cstring>
 #include <iomanip>
+#include <random>
 #include <sstream>
 #include <vector>
 #include <zlib.h>
@@ -198,8 +199,16 @@ namespace ix
             // Set default Content-Type if unspecified
             if (args->extraHeaders.find("Content-Type") == args->extraHeaders.end())
             {
-                ss << "Content-Type: application/x-www-form-urlencoded"
-                   << "\r\n";
+                if (args->multipartBoundary.empty())
+                {
+                    ss << "Content-Type: application/x-www-form-urlencoded"
+                       << "\r\n";
+                }
+                else
+                {
+                    ss << "Content-Type: multipart/form-data; boundary=" << args->multipartBoundary
+                       << "\r\n";
+                }
             }
             ss << "\r\n";
             ss << body;
@@ -597,6 +606,53 @@ namespace ix
         return ss.str();
     }
 
+    std::string HttpClient::serializeHttpFormDataParameters(
+        const std::string& multipartBoundary,
+        const HttpFormDataParameters& httpFormDataParameters,
+        const HttpParameters& httpParameters)
+    {
+        //
+        // --AaB03x
+        // Content-Disposition: form-data; name="submit-name"
+
+        // Larry
+        // --AaB03x
+        // Content-Disposition: form-data; name="foo.txt"; filename="file1.txt"
+        // Content-Type: text/plain
+
+        // ... contents of file1.txt ...
+        // --AaB03x--
+        //
+        std::stringstream ss;
+
+        for (auto&& it : httpFormDataParameters)
+        {
+            ss << "--" << multipartBoundary << "\r\n"
+               << "Content-Disposition:"
+               << " form-data; name=\"" << it.first << "\";"
+               << " filename=\"" << it.first << "\""
+               << "\r\n"
+               << "Content-Type: application/octet-stream"
+               << "\r\n"
+               << "\r\n"
+               << it.second << "\r\n";
+        }
+
+        for (auto&& it : httpParameters)
+        {
+            ss << "--" << multipartBoundary << "\r\n"
+               << "Content-Disposition:"
+               << " form-data; name=\"" << it.first << "\";"
+               << "\r\n"
+               << "\r\n"
+               << it.second << "\r\n";
+        }
+
+        ss << "--" << multipartBoundary << "\r\n";
+
+        return ss.str();
+    }
+
     bool HttpClient::gzipInflate(const std::string& in, std::string& out)
     {
         z_stream inflateState;
@@ -649,4 +705,16 @@ namespace ix
             args->logger(msg);
         }
     }
+
+    std::string HttpClient::generateMultipartBoundary()
+    {
+        std::string str("0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz");
+
+        static std::random_device rd;
+        static std::mt19937 generator(rd());
+
+        std::shuffle(str.begin(), str.end(), generator);
+
+        return str;
+    }
 } // namespace ix

ixwebsocket/IXHttpClient.h

@@ -64,6 +64,13 @@ namespace ix
 
         std::string serializeHttpParameters(const HttpParameters& httpParameters);
 
+        std::string serializeHttpFormDataParameters(
+            const std::string& multipartBoundary,
+            const HttpFormDataParameters& httpFormDataParameters,
+            const HttpParameters& httpParameters = HttpParameters());
+
+        std::string generateMultipartBoundary();
+
         std::string urlEncode(const std::string& value);
 
         const static std::string kPost;

ixwebsocket/IXSocketAppleSSL.cpp

@@ -168,10 +168,32 @@ namespace ix
             SSLSetProtocolVersionMin(_sslContext, kTLSProtocol12);
             SSLSetPeerDomainName(_sslContext, host.c_str(), host.size());
 
-            do
+            if (_tlsOptions.isPeerVerifyDisabled())
             {
-                status = SSLHandshake(_sslContext);
-            } while (errSSLWouldBlock == status || errSSLServerAuthCompleted == status);
+                Boolean option(1);
+                SSLSetSessionOption(_sslContext, kSSLSessionOptionBreakOnServerAuth, option);
+
+                do
+                {
+                    status = SSLHandshake(_sslContext);
+                } while (errSSLWouldBlock == status || errSSLServerAuthCompleted == status);
+
+                if (status == errSSLServerAuthCompleted)
+                {
+                    // proceed with the handshake
+                    do
+                    {
+                        status = SSLHandshake(_sslContext);
+                    } while (errSSLWouldBlock == status || errSSLServerAuthCompleted == status);
+                }
+            }
+            else
+            {
+                do
+                {
+                    status = SSLHandshake(_sslContext);
+                } while (errSSLWouldBlock == status || errSSLServerAuthCompleted == status);
+            }
         }
 
         if (noErr != status)

ixwebsocket/IXSocketFactory.cpp

@@ -31,6 +31,7 @@ namespace ix
                                          std::string& errorMsg,
                                          const SocketTLSOptions& tlsOptions)
     {
+        (void) tlsOptions;
         errorMsg.clear();
         std::shared_ptr<Socket> socket;
 

ixwebsocket/IXSocketSChannel.cpp

@@ -9,6 +9,9 @@
  *
  *  This is the right example to look at:
  *  https://www.codeproject.com/Articles/1000189/A-Working-TCP-Client-and-Server-With-SSL
+ *
+ *  Similar code is available from this git repo
+ *  https://github.com/david-maw/StreamSSL
  */
 #include "IXSocketSChannel.h"
 

ixwebsocket/IXWebSocketTransport.cpp

@@ -543,7 +543,7 @@ namespace ix
             }
 
             // Prevent integer overflow in the next conditional
-            const uint64_t maxFrameSize(1 << 63);
+            const uint64_t maxFrameSize(1ULL << 63);
             if (ws.N > maxFrameSize)
             {
                 return;

ixwebsocket/IXWebSocketVersion.h

@@ -6,4 +6,4 @@
 
 #pragma once
 
-#define IX_WEBSOCKET_VERSION "7.3.1"
+#define IX_WEBSOCKET_VERSION "7.4.3"

makefile

@@ -57,6 +57,7 @@ docker:
 docker_push:
 	docker tag ${IMG} ${LATEST}
 	docker push ${LATEST}
+	docker push ${IMG}
 
 run:
 	docker run --cap-add sys_ptrace --entrypoint=sh -it bsergean/ws:build

test/compatibility/python/websockets/DOCKER_VERSION

@@ -0,0 +1 @@
+0.0.1

test/compatibility/python/websockets/Dockerfile

@@ -0,0 +1,8 @@
+FROM python:3.8.0-alpine3.10
+
+RUN pip install websockets
+COPY ws_proxy.py /usr/bin
+RUN chmod +x /usr/bin/ws_proxy.py
+
+EXPOSE 8765
+CMD ["python", "/usr/bin/ws_proxy.py"]

test/compatibility/python/websockets/makefile

@@ -0,0 +1,20 @@
+.PHONY: docker
+
+NAME   := bsergean/ws_proxy
+TAG    := $(shell cat DOCKER_VERSION)
+IMG    := ${NAME}:${TAG}
+LATEST := ${NAME}:latest
+BUILD  := ${NAME}:build
+
+docker_test:
+	docker build -t ${BUILD} .
+
+docker:
+	git clean -dfx
+	docker build -t ${IMG} .
+	docker tag ${IMG} ${BUILD}
+
+docker_push:
+	docker tag ${IMG} ${LATEST}
+	docker push ${LATEST}
+	docker push ${IMG}

test/compatibility/python/websockets/ws_proxy.py

@@ -0,0 +1,47 @@
+#!/usr/bin/env python3
+
+# websocket proxy
+
+import argparse
+import asyncio
+import websockets
+
+
+async def hello(websocket, path):
+    '''Called whenever a new connection is made to the server'''
+
+    url = REMOTE_URL + path
+    async with websockets.connect(url) as ws:
+        taskA = asyncio.create_task(clientToServer(ws, websocket))
+        taskB = asyncio.create_task(serverToClient(ws, websocket))
+
+        await taskA
+        await taskB
+
+
+async def clientToServer(ws, websocket):
+    async for message in ws:
+        await websocket.send(message)
+
+
+async def serverToClient(ws, websocket):
+    async for message in websocket:
+        await ws.send(message)
+
+
+if __name__ == '__main__':
+    parser = argparse.ArgumentParser(description='websocket proxy.')
+    parser.add_argument('--host', help='Host to bind to.',
+                        default='localhost')
+    parser.add_argument('--port', help='Port to bind to.',
+                        default=8765)
+    parser.add_argument('--remote_url', help='Remote websocket url',
+                        default='ws://localhost:8767')
+    args = parser.parse_args()
+
+    REMOTE_URL = args.remote_url
+
+    start_server = websockets.serve(hello, args.host, args.port)
+
+    asyncio.get_event_loop().run_until_complete(start_server)
+    asyncio.get_event_loop().run_forever()

ws/CMakeLists.txt

@@ -56,6 +56,7 @@ add_executable(ws
   ws_httpd.cpp
   ws_autobahn.cpp
   ws_proxy_server.cpp
+  ws_sentry_minidump_upload.cpp
   ws.cpp)
 
 target_link_libraries(ws ixsnake)

ws/ws.cpp

@@ -73,6 +73,10 @@ int main(int argc, char** argv)
     std::string appsConfigPath("appsConfig.json");
     std::string subprotocol;
     std::string remoteHost;
+    std::string minidump;
+    std::string metadata;
+    std::string project;
+    std::string key;
     ix::SocketTLSOptions tlsOptions;
     std::string ciphers;
     std::string redirectUrl;
@@ -210,7 +214,7 @@ int main(int argc, char** argv)
     cobraSubscribeApp->add_option("--endpoint", endpoint, "Endpoint")->required();
     cobraSubscribeApp->add_option("--rolename", rolename, "Role name")->required();
     cobraSubscribeApp->add_option("--rolesecret", rolesecret, "Role secret")->required();
-    cobraSubscribeApp->add_option("channel", channel, "Channel")->required();
+    cobraSubscribeApp->add_option("--channel", channel, "Channel")->required();
     cobraSubscribeApp->add_option("--pidfile", pidfile, "Pid file");
     cobraSubscribeApp->add_option("--filter", filter, "Stream SQL Filter");
     cobraSubscribeApp->add_flag("-q", quiet, "Quiet / only display stats");
@@ -220,7 +224,7 @@ int main(int argc, char** argv)
     cobraPublish->add_option("--endpoint", endpoint, "Endpoint")->required();
     cobraPublish->add_option("--rolename", rolename, "Role name")->required();
     cobraPublish->add_option("--rolesecret", rolesecret, "Role secret")->required();
-    cobraPublish->add_option("channel", channel, "Channel")->required();
+    cobraPublish->add_option("--channel", channel, "Channel")->required();
     cobraPublish->add_option("--pidfile", pidfile, "Pid file");
     cobraPublish->add_option("path", path, "Path to the file to send")
         ->required()
@@ -232,7 +236,7 @@ int main(int argc, char** argv)
     cobraMetricsPublish->add_option("--endpoint", endpoint, "Endpoint");
     cobraMetricsPublish->add_option("--rolename", rolename, "Role name");
     cobraMetricsPublish->add_option("--rolesecret", rolesecret, "Role secret");
-    cobraMetricsPublish->add_option("channel", channel, "Channel")->required();
+    cobraMetricsPublish->add_option("--channel", channel, "Channel")->required();
     cobraMetricsPublish->add_option("--pidfile", pidfile, "Pid file");
     cobraMetricsPublish->add_option("path", path, "Path to the file to send")
         ->required()
@@ -311,6 +315,13 @@ int main(int argc, char** argv)
     proxyServerApp->add_option("--remote_host", remoteHost, "Remote Hostname");
     proxyServerApp->add_flag("-v", verbose, "Verbose");
 
+    CLI::App* minidumpApp = app.add_subcommand("upload_minidump", "Upload a minidump to sentry");
+    minidumpApp->add_option("--minidump", minidump, "Minidump path")->check(CLI::ExistingPath);
+    minidumpApp->add_option("--metadata", metadata, "Hostname")->check(CLI::ExistingPath);
+    minidumpApp->add_option("--project", project, "Sentry Project")->required();
+    minidumpApp->add_option("--key", key, "Sentry Key")->required();
+    minidumpApp->add_flag("-v", verbose, "Verbose");
+
     CLI11_PARSE(app, argc, argv);
 
     // pid file handling
@@ -453,6 +464,10 @@ int main(int argc, char** argv)
     {
         ret = ix::ws_proxy_server_main(port, hostname, tlsOptions, remoteHost, verbose);
     }
+    else if (app.got_subcommand("upload_minidump"))
+    {
+        ret = ix::ws_sentry_minidump_upload(metadata, minidump, project, key, verbose);
+    }
     else if (version)
     {
         std::cout << "ws " << ix::userAgent() << std::endl;

ws/ws.h

@@ -148,4 +148,10 @@ namespace ix
                              const ix::SocketTLSOptions& tlsOptions,
                              const std::string& remoteHost,
                              bool verbose);
+
+    int ws_sentry_minidump_upload(const std::string& metadataPath,
+                                  const std::string& minidump,
+                                  const std::string& project,
+                                  const std::string& key,
+                                  bool verbose);
 } // namespace ix

ws/ws_cobra_subscribe.cpp

@@ -99,7 +99,7 @@ namespace ix
 
         while (true)
         {
-            std::chrono::duration<double, std::milli> duration(10);
+            auto duration = std::chrono::seconds(1);
             std::this_thread::sleep_for(duration);
         }
 

ws/ws_cobra_to_sentry.cpp

@@ -81,6 +81,29 @@ namespace ix
 
                 auto ret = sentryClient.send(msg, verbose);
                 HttpResponsePtr response = ret.first;
+
+                if (verbose)
+                {
+                    for (auto it : response->headers)
+                    {
+                        spdlog::info("{}: {}", it.first, it.second);
+                    }
+
+                    spdlog::info("Upload size: {}", response->uploadSize);
+                    spdlog::info("Download size: {}", response->downloadSize);
+
+                    spdlog::info("Status: {}", response->statusCode);
+                    if (response->errorCode != HttpErrorCode::Ok)
+                    {
+                        spdlog::info("error message: {}", response->errorMsg);
+                    }
+
+                    if (response->headers["Content-Type"] != "application/octet-stream")
+                    {
+                        spdlog::info("payload: {}", response->payload);
+                    }
+                }
+
                 if (response->statusCode != 200)
                 {
                     spdlog::error("Error sending data to sentry: {}", response->statusCode);

ws/ws_proxy_server.cpp

@@ -65,20 +65,21 @@ namespace ix
                 if (msg->type == ix::WebSocketMessageType::Open)
                 {
                     std::cerr << "New connection" << std::endl;
-                    std::cerr << "id: " << state->getId() << std::endl;
+                    std::cerr << "server id: " << state->getId() << std::endl;
                     std::cerr << "Uri: " << msg->openInfo.uri << std::endl;
                     std::cerr << "Headers:" << std::endl;
                     for (auto it : msg->openInfo.headers)
                     {
                         std::cerr << it.first << ": " << it.second << std::endl;
                     }
-                    state->setConnected();
                 }
                 else if (msg->type == ix::WebSocketMessageType::Close)
                 {
                     std::cerr << "Closed connection"
                               << " code " << msg->closeInfo.code << " reason "
                               << msg->closeInfo.reason << std::endl;
+                    webSocket->close(msg->closeInfo.code, msg->closeInfo.reason);
+                    state->setTerminated();
                 }
                 else if (msg->type == ix::WebSocketMessageType::Error)
                 {
@@ -88,7 +89,6 @@ namespace ix
                     ss << "Wait time(ms): " << msg->errorInfo.wait_time << std::endl;
                     ss << "HTTP Status: " << msg->errorInfo.http_status << std::endl;
                     std::cerr << ss.str();
-                    webSocket->close(msg->closeInfo.code, msg->closeInfo.reason);
                 }
                 else if (msg->type == ix::WebSocketMessageType::Message)
                 {
@@ -108,7 +108,7 @@ namespace ix
                 if (msg->type == ix::WebSocketMessageType::Open)
                 {
                     std::cerr << "New connection" << std::endl;
-                    std::cerr << "id: " << state->getId() << std::endl;
+                    std::cerr << "client id: " << state->getId() << std::endl;
                     std::cerr << "Uri: " << msg->openInfo.uri << std::endl;
                     std::cerr << "Headers:" << std::endl;
                     for (auto it : msg->openInfo.headers)
@@ -124,10 +124,12 @@ namespace ix
 
                     // we should sleep here for a bit until we've established the
                     // connection with the remote server
-                    while (!state->isConnected())
+                    while (state->webSocket().getReadyState() != ReadyState::Open)
                     {
+                        std::cerr << "waiting for server connection establishment" << std::endl;
                         std::this_thread::sleep_for(std::chrono::milliseconds(10));
                     }
+                    std::cerr << "server connection established" << std::endl;
                 }
                 else if (msg->type == ix::WebSocketMessageType::Close)
                 {

ws/ws_sentry_minidump_upload.cpp

@@ -0,0 +1,105 @@
+/*
+ *  ws_sentry_minidump_upload.cpp
+ *  Author: Benjamin Sergeant
+ *  Copyright (c) 2019 Machine Zone, Inc. All rights reserved.
+ */
+
+#include <fstream>
+#include <ixsentry/IXSentryClient.h>
+#include <jsoncpp/json/json.h>
+#include <spdlog/spdlog.h>
+#include <sstream>
+
+
+namespace
+{
+    // Assume the file exists
+    std::string readBytes(const std::string& path)
+    {
+        std::vector<uint8_t> memblock;
+        std::ifstream file(path);
+
+        file.seekg(0, file.end);
+        std::streamoff size = file.tellg();
+        file.seekg(0, file.beg);
+
+        memblock.resize(size);
+
+        file.read((char*) &memblock.front(), static_cast<std::streamsize>(size));
+
+        std::string bytes(memblock.begin(), memblock.end());
+        return bytes;
+    }
+} // namespace
+
+namespace ix
+{
+    int ws_sentry_minidump_upload(const std::string& metadataPath,
+                                  const std::string& minidump,
+                                  const std::string& project,
+                                  const std::string& key,
+                                  bool verbose)
+    {
+        SentryClient sentryClient((std::string()));
+
+        // Read minidump file from disk
+        std::string minidumpBytes = readBytes(minidump);
+
+        // Read json data
+        std::string sentryMetadata = readBytes(metadataPath);
+
+
+        std::atomic<bool> done(false);
+
+        sentryClient.uploadMinidump(
+            sentryMetadata,
+            minidumpBytes,
+            project,
+            key,
+            verbose,
+            [verbose, &done](const HttpResponsePtr& response) {
+                if (verbose)
+                {
+                    for (auto it : response->headers)
+                    {
+                        spdlog::info("{}: {}", it.first, it.second);
+                    }
+
+                    spdlog::info("Upload size: {}", response->uploadSize);
+                    spdlog::info("Download size: {}", response->downloadSize);
+
+                    spdlog::info("Status: {}", response->statusCode);
+                    if (response->errorCode != HttpErrorCode::Ok)
+                    {
+                        spdlog::info("error message: {}", response->errorMsg);
+                    }
+
+                    if (response->headers["Content-Type"] != "application/octet-stream")
+                    {
+                        spdlog::info("payload: {}", response->payload);
+                    }
+                }
+
+                if (response->statusCode != 200)
+                {
+                    spdlog::error("Error sending data to sentry: {}", response->statusCode);
+                    spdlog::error("Status: {}", response->statusCode);
+                    spdlog::error("Response: {}", response->payload);
+                }
+                else
+                {
+                    spdlog::info("Event sent to sentry");
+                }
+
+                done = true;
+            });
+
+        while (!done)
+        {
+            std::chrono::duration<double, std::milli> duration(10);
+            std::this_thread::sleep_for(duration);
+        }
+
+        return 0;
+    }
+} // namespace ix