tag:status.tardis.dev,2005:/historyTardis.dev Status - Incident History2024-03-29T05:23:12ZTardis.devtag:status.tardis.dev,2005:Incident/202763242024-03-16T21:03:07Z2024-03-16T21:03:07ZDelayed CSV data exports for 2023-03-15 for coinbase, kucoin and kucoin-futures<p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>21:03</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>08:48</var> UTC</small><br><strong>Update</strong> - ETA 4PM UTC.</p><p><small>Mar <var data-var='date'>16</var>, <var data-var='time'>08:40</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p>tag:status.tardis.dev,2005:Incident/201925312024-03-09T07:31:13Z2024-03-09T07:31:13ZDelayed CSV data exports for 2023-03-07 for coinbase, okex-swap and kucoin<p><small>Mar <var data-var='date'> 9</var>, <var data-var='time'>07:31</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 8</var>, <var data-var='time'>12:44</var> UTC</small><br><strong>Update</strong> - coinbase and okex-swap exports finished.</p><p><small>Mar <var data-var='date'> 8</var>, <var data-var='time'>11:03</var> UTC</small><br><strong>Monitoring</strong> - ETA for coinbase: 2PM UTC<br />ETA for kucoin: 8 PM UTC<br />ETA for okex-swap: 1 PM UTC</p>tag:status.tardis.dev,2005:Incident/201826862024-03-08T10:57:14Z2024-03-08T10:57:14ZDelayed CSV data exports for 2023-03-06 for multiple exchanges<p><small>Mar <var data-var='date'> 8</var>, <var data-var='time'>10:57</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>15:19</var> UTC</small><br><strong>Update</strong> - remaining CSV exports:<br />- kucoin<br />- delta</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>13:47</var> UTC</small><br><strong>Update</strong> - remaining CSV exports:<br /><br />- coinbase<br />- kucoin<br />- delta</p><p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>07:22</var> UTC</small><br><strong>Monitoring</strong> - delayed CSV exports for:<br />- deribit<br />- okex<br />- okex-swap<br />- coinbase<br />- kucoin<br />- kucoin-futures<br />- delta</p>tag:status.tardis.dev,2005:Incident/201736682024-03-07T05:40:19Z2024-03-07T05:40:19ZDelayed CSV data exports for 2023-03-05 for multiple exchanges<p><small>Mar <var data-var='date'> 7</var>, <var data-var='time'>05:40</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>14:05</var> UTC</small><br><strong>Update</strong> - exports finished for all exchanges except:<br />- kucoin<br />- delta</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>13:05</var> UTC</small><br><strong>Update</strong> - data exports for:<br />- okex<br />- okex-swap<br />- bybit <br /><br />have finished</p><p><small>Mar <var data-var='date'> 6</var>, <var data-var='time'>08:04</var> UTC</small><br><strong>Identified</strong> - Affected exchanges:<br />- okex<br />- okex-swap<br />- coinbase<br />- bybit<br />- kucoin<br />- kucoin-futures<br />- delta</p>tag:status.tardis.dev,2005:Incident/201243262024-03-02T17:35:20Z2024-03-02T17:35:20ZDelayed CSV data exports for 2023-02-29 for multiple exchanges<p><small>Mar <var data-var='date'> 2</var>, <var data-var='time'>17:35</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>10:03</var> UTC</small><br><strong>Update</strong> - remaining affected exchanges:<br />- kucoin</p><p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>07:04</var> UTC</small><br><strong>Monitoring</strong> - affected exchanges:<br />- binance-futures<br />- binance-european-options<br />- okex-swap<br />- kucoin<br />- kucoin-futures</p>tag:status.tardis.dev,2005:Incident/201039812024-02-29T18:25:56Z2024-02-29T18:25:56ZDelayed CSV data exports for 2023-02-28 for multiple exchanges<p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>18:25</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>09:21</var> UTC</small><br><strong>Update</strong> - exports finished already for:<br />- coinbase <br />- binance<br />- binance-futures<br />- okex<br />- okex-swap</p><p><small>Feb <var data-var='date'>29</var>, <var data-var='time'>06:15</var> UTC</small><br><strong>Monitoring</strong> - affected exchanges:<br />- coinbase <br />- binance<br />- binance-futures<br />- okex<br />- okex-swap<br />- kucoin<br />- kucoin-futures</p>tag:status.tardis.dev,2005:Incident/200555572024-02-23T12:28:58Z2024-02-23T12:28:58ZDelayed CSV data exports for 2023-02-2 for multiple exchanges<p><small>Feb <var data-var='date'>23</var>, <var data-var='time'>12:28</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'>23</var>, <var data-var='time'>08:50</var> UTC</small><br><strong>Identified</strong> - There's an issue with slow uploads happening currently for one of the cloud storages that are backing up data export, affected exchanges:<br />- binance-futures<br />- binance<br />- okex-swap<br />- coinbase<br />- gate-io<br />- kraken<br />- okex</p>tag:status.tardis.dev,2005:Incident/198557332024-01-29T09:23:30Z2024-01-29T09:23:30Ztardis-machine v3.28.0 update<p><small>Jan <var data-var='date'>29</var>, <var data-var='time'>09:23</var> UTC</small><br><strong>Resolved</strong> - add kucoin-futures support</p>tag:status.tardis.dev,2005:Incident/197641202024-01-19T13:11:20Z2024-01-19T13:11:20ZDelayed CSV data exports for 2023-01-18 for gate-io and delta exchanges<p><small>Jan <var data-var='date'>19</var>, <var data-var='time'>13:11</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>19</var>, <var data-var='time'>11:20</var> UTC</small><br><strong>Update</strong> - gate-io export finished</p><p><small>Jan <var data-var='date'>19</var>, <var data-var='time'>10:52</var> UTC</small><br><strong>Monitoring</strong> - gate-io ETA: 12:00 UTC<br />delta ETA: 15:00 UTC</p>tag:status.tardis.dev,2005:Incident/197485332024-01-19T08:54:16Z2024-01-19T08:54:16ZDelayed CSV data exports for 2023-01-17<p><small>Jan <var data-var='date'>19</var>, <var data-var='time'>08:54</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>18</var>, <var data-var='time'>07:03</var> UTC</small><br><strong>Identified</strong> - This was caused by issue introduced in lated release, fix is being worked on.</p>tag:status.tardis.dev,2005:Incident/196870502024-01-14T09:42:03Z2024-01-14T09:42:03ZDelayed CSV data exports for kucoin for 2023-01-11<p><small>Jan <var data-var='date'>14</var>, <var data-var='time'>09:42</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jan <var data-var='date'>12</var>, <var data-var='time'>11:00</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p>tag:status.tardis.dev,2005:Incident/194956402023-12-22T06:51:16Z2023-12-22T06:51:16ZDelayed CSV data exports for kraken, gate-io and bybit for 2023-12-20<p><small>Dec <var data-var='date'>22</var>, <var data-var='time'>06:51</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Dec <var data-var='date'>21</var>, <var data-var='time'>07:22</var> UTC</small><br><strong>Monitoring</strong> - A fix has been implemented and we are monitoring the results.</p>tag:status.tardis.dev,2005:Incident/189219562023-10-25T09:41:54Z2023-10-25T09:41:54ZRe-exported binance futures liquidations data ( use Order Last Filled Quantity for amount field and price for last price field)<p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>09:41</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p>tag:status.tardis.dev,2005:Incident/189218312023-10-25T09:24:43Z2023-10-25T09:24:43ZRe-exported okx (spot,futures,swap,options) trades CSV data since 2023-10-19 (use trades-all channel as a normalized trades data source for okx)<p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>09:24</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p>tag:status.tardis.dev,2005:Incident/189216892023-10-25T09:11:02Z2023-10-25T09:11:02ZRe-exported woo-x trades CSV data (exclude trades with source==0 on woo-x)<p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>09:11</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p>tag:status.tardis.dev,2005:Incident/189216182023-10-25T09:02:24Z2023-10-25T09:04:30Ztardis-machine v3.25.0 update<p><small>Oct <var data-var='date'>25</var>, <var data-var='time'>09:02</var> UTC</small><br><strong>Resolved</strong> - <b><br />- use trades-all channel as a normalized trades data source for okx<br />- use new streams.deribit.com url for Deribit WS market data feeds<br />- exclude trades with source==0 on woo-x<br />- for binance futures liquidations use Order Last Filled Quantity for amount field<br /></b></p>tag:status.tardis.dev,2005:Incident/184669742023-09-07T09:26:00Z2023-09-11T09:27:37ZRe-exported kraken futures funding related CSV data<p><small>Sep <var data-var='date'> 7</var>, <var data-var='time'>09:26</var> UTC</small><br><strong>Resolved</strong> - use kraken futures relative funding rate post September 29, 2022 -<br />https://support.kraken.com/hc/en-us/articles/9618146737172-Perpetual-Contracts-Funding-Rate-Method-Prior-to-September-29-2022</p>tag:status.tardis.dev,2005:Incident/180943932023-08-09T12:14:50Z2023-08-09T12:14:50ZDelayed CSV data exports for 2023-08-08 for upbit exchange<p><small>Aug <var data-var='date'> 9</var>, <var data-var='time'>12:14</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Aug <var data-var='date'> 9</var>, <var data-var='time'>11:11</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.tardis.dev,2005:Incident/174857502023-06-07T05:43:53Z2023-06-07T05:43:53ZDelayed CSV data exports for 2023-06-05<p><small>Jun <var data-var='date'> 7</var>, <var data-var='time'>05:43</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>15:21</var> UTC</small><br><strong>Update</strong> - Exports finished for:<br />kraken<br />crypto-com<br /><br />Still pending exports:<br />kucoin<br />gate-io<br /><br />ETA 8PM UTC.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>14:02</var> UTC</small><br><strong>Update</strong> - Exports finished for:<br />okex<br />bybit-options<br />bitfinex<br />delta<br />bybit<br /><br />Still pending exports:<br />kraken<br />kucoin<br />gate-io<br />crypto-com</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>11:10</var> UTC</small><br><strong>Monitoring</strong> - Following exchanges CSV exports are still pending: <br />okex<br />huobi<br />bitfinex<br />kraken<br />bybit<br />bybit-options<br />kucoin<br />delta<br />gate-io<br />crypto-com<br /><br />Remaining exchanges exports finished.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>05:07</var> UTC</small><br><strong>Update</strong> - Current ETA is 11AM UTC.</p><p><small>Jun <var data-var='date'> 6</var>, <var data-var='time'>04:34</var> UTC</small><br><strong>Investigating</strong> - We are currently investigating this issue.</p>tag:status.tardis.dev,2005:Incident/174210682023-05-31T05:14:40Z2023-05-31T05:14:40Ztardis-machine v3.22.0: support coinbase authenticated channels and fallback to level2_batch for l2 data for non authenticated connections<p><small>May <var data-var='date'>31</var>, <var data-var='time'>05:14</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>30</var>, <var data-var='time'>13:30</var> UTC</small><br><strong>Investigating</strong> - Coinbase real-time feeds now by default use level2_batch for l2 data, but when authenticated will use level2 (non batched real-time channel). This is caused by following incoming Coinbase API changes:<br />https://docs.cloud.coinbase.com/exchange/docs/upcoming-changes<br />ENFORCED AUTHENTICATION<br />2023-MAY-03<br />Effective July 1, 2023, the following WebSocket feeds on Coinbase Exchange will require authentication:<br /><br />Full Channel<br />Level2 Channel<br />Level3 Channel<br /><br /><br /><br />In order to authenticate Coinbase API connections following environment variables need to be set:<br />- COINBASE_API_KEY<br />- COINBASE_API_SECRET<br />- COINBASE_API_PASSPHRASE<br /><br />https://github.com/tardis-dev/tardis-node/commit/501a3eaa9ba0ca376230cb384966cfa630fc33b3</p>tag:status.tardis.dev,2005:Incident/174067662023-05-29T13:06:26Z2023-05-29T13:06:26ZDelayed CSV data exports for okex, okex-swap, coinbase, binance and binance-futures (2023-05-28)<p><small>May <var data-var='date'>29</var>, <var data-var='time'>13:06</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>May <var data-var='date'>29</var>, <var data-var='time'>12:01</var> UTC</small><br><strong>Update</strong> - ETA for okex spot export is 2PM UTC today.</p><p><small>May <var data-var='date'>29</var>, <var data-var='time'>11:59</var> UTC</small><br><strong>Update</strong> - coinbase and binance-futures exports finished.</p><p><small>May <var data-var='date'>29</var>, <var data-var='time'>11:09</var> UTC</small><br><strong>Update</strong> - okex-swap and binance exports finished.</p><p><small>May <var data-var='date'>29</var>, <var data-var='time'>07:15</var> UTC</small><br><strong>Identified</strong> - The issue has been identified and a fix is being implemented.</p>tag:status.tardis.dev,2005:Incident/167569492023-04-05T07:52:08Z2023-04-05T07:52:08ZBybit data collection incident<p><small>Apr <var data-var='date'> 5</var>, <var data-var='time'>07:52</var> UTC</small><br><strong>Resolved</strong> - Missing feed data from 2023-04-05T00:00:00.000Z to 2023-04-05T05:50:00.000Z due to deployment bug that was introduced when switching to Bybit API v5.</p>tag:status.tardis.dev,2005:Incident/162713682023-03-01T08:38:15Z2023-03-01T08:38:15ZMissing okx data due to AWS PrivateLink VPC colocation disabled by OKX<p><small>Mar <var data-var='date'> 1</var>, <var data-var='time'>08:38</var> UTC</small><br><strong>Resolved</strong> - OKX feeds as mentioned were switched to use 'books' channel (100ms aggregated updates) for order book related data.<br />CSV downloads were exported using 'books' as data source.<br />Still waiting on re-enabling 'books-l2-tbt' feed for Tardis.</p><p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>21:14</var> UTC</small><br><strong>Identified</strong> - Today at 2023-02-24T09:50:00.000Z data collection stopped working for OKX due to AWS PrivateLink VPC colocation disabled by OKX for Tardis. <br /><br />Waiting on re-enabling it by OKX team and in the meanwhile since 2023-02-24T11:20:00.000Z data is being collected without VPC colo setup, meaning no order book tick level data is being collected (no books-l2-tbt channel).<br /><br />Starting at 2023-02-24T21:07:00.000Z since no resolution on OKX side instead of books-l2-tbt, books channel is being used (100ms aggregated updates).</p>tag:status.tardis.dev,2005:Incident/162713572023-02-24T08:30:00Z2023-02-24T21:12:37ZMissing okx data due to AWS PrivateLink VPC colocation disabled by OKX for Tardis<p><small>Feb <var data-var='date'>24</var>, <var data-var='time'>08:30</var> UTC</small><br><strong>Resolved</strong> - Today at 2023-02-24T09:50:00.000Z data collection stopped working for OKX due to AWS PrivateLink VPC colocation disabled by OKX for Tardis. <br />Waiting on re-enabling it by OKX team in the meanwhile since 2023-02-24T11:20:00.000Z data is being collected without VPC colo setup, meaning no order book tick level data is being collected (no books-l2-tbt channel).<br />Starting at 2023-02-24T21:07:00.000Z since no resolution on OKX side instead of books-l2-tbt, books channel is being used (100ms aggregated updates).</p>tag:status.tardis.dev,2005:Incident/160168572023-02-03T15:58:27Z2023-02-03T15:58:27ZDelayed CSV data exports for 2023-02-02 due to APIs outage<p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>15:58</var> UTC</small><br><strong>Resolved</strong> - This incident has been resolved.</p><p><small>Feb <var data-var='date'> 3</var>, <var data-var='time'>12:43</var> UTC</small><br><strong>Monitoring</strong> - due to https://status.tardis.dev/incidents/zyfn9p37nvgn CSV exports are delayed.</p>