kapitan/IXWebSocket
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

redis conf slides

Browse Source
This commit is contained in:
Benjamin Sergeant 2019-03-28 14:17:19 -07:00
parent 3004422cb6
commit c865d64608
2 changed files with 50 additions and 63 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
doc/redis_conf_2019/neo_map.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 113 KiB

113
doc/redis_conf_2019/slides.html
View File

@ -60,22 +60,10 @@
<body> <body>
<textarea id="source"> <textarea id="source">
class: center, middle
# RedisConf 2019
Real-Time Health Analytics with WebSockets, Python 3 and Redis PubSub
---
name: first name: first
--- ---
name: ten-years
---
class: center, middle class: center, middle
# Hello world # Hello world
@ -421,7 +409,7 @@ To be able to scale horizontally we run multiple redis instances, and we select
| | | |
+---------->+ | +-------------+ +---------->+ | +-------------+
| SELECT ... | | few messages| | SELECT ... | | few messages|
High frenquency | +---->+ received | High frequency | +---->+ received |
message | (subset | | | message | (subset | | |
| of data) | +-------------+ | of data) | +-------------+
+---------->+ | +---------->+ |
@ -431,11 +419,44 @@ To be able to scale horizontally we run multiple redis instances, and we select
--- ---
# Deployment # Example Stream SQL expressions
- OpenShift (Redhat tooling around Kubernete) ## HTTP Errors
- In memory redis config.
- It works (tm) ```
SELECT data.url,data.status FROM \`engine_net_request_id\`
WHERE data.status != 200 AND
data.url LIKE 'why'
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_one_too', 'data.status': 404}
```
## CDN requests
```
SELECT data.url,data.duration_ms FROM \`engine_net_request_id\`
WHERE data.url LIKE '.png'
{'data.url': 'https://big.cdn.provider/asset_1.png', 'data.duration_ms': 28}}
{'data.url': 'https://big.cdn.provider/asset_2.png', 'data.duration_ms': 1008}}
{'data.url': 'https://other.cdn.provider/asset_2.png', 'data.duration_ms': 512}}
```
???
We used this to deprecate sending low res assets to old devices, as very few of them remained in use, and it would speed up our deployment process, and reduce our CDNs costs to do so.
---
# Deployed with OpenShift
- 60 python nodes (1CPU + 4G) dedicated to publishers
- 10 python nodes (1CPU + 4G) dedicated to subscribers
- 5 redis nodes (512M ?)
``` ```
redis1-1-gbjvt 1/1 Running 2 145d redis1-1-gbjvt 1/1 Running 2 145d
@ -446,17 +467,17 @@ redis5-1-z6262 1/1 Running 0 52d
``` ```
``` ```
cobra-44-4xfbw 1/1 Running 0 1h cobra-44-4xfbw 1/1 Running 0 12d
cobra-44-58vch 1/1 Running 0 1h cobra-44-58vch 1/1 Running 0 12d
cobra-44-5m4ff 1/1 Running 0 1h cobra-44-5m4ff 1/1 Running 0 12d
cobra-44-6vvk5 1/1 Running 0 1h cobra-44-6vvk5 1/1 Running 0 12d
cobra-44-7bmcx 1/1 Running 0 1h ...
cobra-44-85q78 1/1 Running 0 1h
cobra-subscribers-30-7bpqh 1/1 Running 0 1h cobra-subscribers-30-7bpqh 1/1 Running 0 12d
cobra-subscribers-30-fd477 1/1 Running 0 1h cobra-subscribers-30-fd477 1/1 Running 0 12d
cobra-subscribers-30-h5ntn 1/1 Running 0 1h cobra-subscribers-30-h5ntn 1/1 Running 0 12d
cobra-subscribers-30-hlcd6 1/1 Running 0 1h cobra-subscribers-30-hlcd6 1/1 Running 0 12d
...
``` ```
??? ???
@ -541,10 +562,10 @@ Good property of a flexible system.
# Neo # Neo
- Interactive real-time shell (good for non programmers) - Real time dashboard
- Search/Filter with StreamSQL - Search/Filter with StreamSQL
![Alt text](neo.png) ![Alt text](neo_map.png)
--- ---
@ -611,40 +632,6 @@ class: center, middle
??? ???
I will not go through this in the presentation I will not go through this in the presentation
---
# Example Stream SQL expressions
## HTTP Errors
```
SELECT data.url,data.status FROM \`engine_net_request_id\`
WHERE data.status != 200 AND
data.url LIKE 'why'
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_request', 'data.status': 404}
{'data.url': 'https://why_are_we_still_making/this_one_too', 'data.status': 404}
```
## CDN requests
```
SELECT data.url,data.duration_ms FROM \`engine_net_request_id\`
WHERE data.url LIKE '.png'
{'data.url': 'https://big.cdn.provider/asset_1.png', 'data.duration_ms': 28}}
{'data.url': 'https://big.cdn.provider/asset_2.png', 'data.duration_ms': 1008}}
{'data.url': 'https://other.cdn.provider/asset_2.png', 'data.duration_ms': 512}}
```
???
We used this to deprecate sending low res assets to old devices, as very few of them remained in use, and it would speed up our deployment process, and reduce our CDNs costs to do so.
--- ---
class: center, middle class: center, middle