Seatable suddenly stopped working

Your Setup:

  • Hardware or environment you use:
    Docker setup

  • SeaTable Edition (Developer or Enterprise)
    Developer

  • SeaTable Version (only necessary for self-hosted):
    {"version":"5.1.0","edition":"developer edition"}

I have a Seatable instance which I use from time to time with not much data (maybe a a couple of thousands of lines).

Today it suddenly stopped working. I have daily n8n backups for Seatable and I noticed that the backup failed today.

I can login without a problem but as soon as I open a Base, it shows me the following error message

I did try to troubleshoot myself but no luck. Here are some error messages that might be relevant.

dtable-server.error.log

2025/02/05 12:38:09 [error] 60#60: *1766 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /dtable-server/dtables/3d1eb410-9228-4b19-a209-835af6592bde?lang=en HTTP/1.1", upstream: "http://127.0.0.1:5000/dtables/3d1eb410-9228-4b19-a209-835af6592bde?lang=en", host: "tables.domain.com", referrer: "https://tables.domain.com/workspace/4/dtable/Administrim%202025/"
2025/02/05 12:38:17 [error] 60#60: *1774 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /dtable-server/dtables/50dac823-6a03-44ee-b12d-42df3258fbd9?lang=en HTTP/1.1", upstream: "http://127.0.0.1:5000/dtables/50dac823-6a03-44ee-b12d-42df3258fbd9?lang=en", host: "tables.domain.com", referrer: "https://tables.domain.com/workspace/4/dtable/Administrim%202024/"
2025/02/05 12:40:26 [error] 58#58: *1798 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /dtable-server/dtables/3d1eb410-9228-4b19-a209-835af6592bde?lang=en HTTP/1.1", upstream: "http://127.0.0.1:5000/dtables/3d1eb410-9228-4b19-a209-835af6592bde?lang=en", host: "tables.domain.com", referrer: "https://tables.domain.com/workspace/4/dtable/Administrim%202025/"
2025/02/05 12:40:29 [error] 53#53: *1808 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /dtable-server/dtables/50dac823-6a03-44ee-b12d-42df3258fbd9?lang=en HTTP/1.1", upstream: "http://127.0.0.1:5000/dtables/50dac823-6a03-44ee-b12d-42df3258fbd9?lang=en", host: "tables.domain.com", referrer: "https://tables.domain.com/workspace/4/dtable/Administrim%202024/"

dtable-web.error.log

2025/02/05 12:11:27 [error] 53#53: *17 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "tables.domain.com", referrer: "https://tables.domain.com/"
2025/02/05 12:11:27 [error] 53#53: *19 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "tables.domain.com", referrer: "https://tables.domain.com/sys/users/"
2025/02/05 12:11:27 [error] 53#53: *21 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "tables.domain.com", referrer: "https://tables.domain.com/"
2025/02/05 12:11:27 [error] 53#53: *23 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "tables.domain.com", referrer: "https://tables.domain.com/"
2025/02/05 12:11:27 [error] 53#53: *25 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "tables.domain.com", referrer: "https://tables.domain.com/sys/users/"
2025/02/05 12:11:27 [error] 53#53: *27 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "tables.domain.com", referrer: "https://tables.domain.com/"
2025/02/05 12:11:27 [error] 53#53: *29 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8000/favicon.ico", host: "tables.domain.com", referrer: "https://tables.domain.com/"
2025/02/05 12:11:27 [error] 53#53: *31 connect() failed (111: Connection refused) while connecting to upstream, client: 172.23.0.1, server: 127.0.0.1, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8000/", host: "tables.domain.com", referrer: "https://tables.domain.com/sys/users/"

dtable_events_io.log

Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connectionpool.py", line 715, in urlopen
    httplib_response = self._make_request(
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connectionpool.py", line 416, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connection.py", line 244, in request
    super(HTTPConnection, self).request(method, url, body=body, headers=headers)
  File "/usr/lib/python3.8/http/client.py", line 1256, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1302, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1251, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib/python3.8/http/client.py", line 1011, in _send_output
    self.send(msg)
  File "/usr/lib/python3.8/http/client.py", line 951, in send
    self.connect()
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connection.py", line 205, in connect
    conn = self._new_conn()
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connection.py", line 186, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f9251054fa0>: Failed to establish a new connection: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /dtables/ee67013a-dc86-48c2-b498-27e61f96eb7f?from=dtable_events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9251054fa0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-events/dtable_events/dtable_io/__init__.py", line 77, in get_dtable_export_content
    prepare_dtable_json_from_memory(workspace_id, dtable_uuid, username)
  File "/opt/seatable/seatable-server-latest/dtable-events/dtable_events/dtable_io/utils.py", line 136, in prepare_dtable_json_from_memory
    json_content = dtable_server_api.get_base()
  File "/opt/seatable/seatable-server-latest/dtable-events/dtable_events/utils/dtable_server_api.py", line 169, in get_base
    response = requests.get(url, headers=self.headers, timeout=self.timeout)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /dtables/ee67013a-dc86-48c2-b498-27e61f96eb7f?from=dtable_events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9251054fa0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-events/dtable_events/dtable_io/task_manager.py", line 417, in handle_task
    task[0](*task[1])
  File "/opt/seatable/seatable-server-latest/dtable-events/dtable_events/dtable_io/__init__.py", line 80, in get_dtable_export_content
    raise Exception('prepare dtable json failed. ERROR: {}'.format(e))
Exception: prepare dtable json failed. ERROR: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /dtables/ee67013a-dc86-48c2-b498-27e61f96eb7f?from=dtable_events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9251054fa0>: Failed to establish a new connection: [Errno 111] Connection refused'))
2025-02-05 11:00:01,100 [ERROR] Failed to handle task 7b825c33-6293-4e8d-b0ef-d8e22f25aa7b <function get_dtable_export_content at 0x7f9256079f70>, error: prepare dtable json failed. ERROR: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /dtables/ee67013a-dc86-48c2-b498-27e61f96eb7f?from=dtable_events (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f9251054fa0>: Failed to establish a new connection: [Errno 111] Connection refused'))

dtable_web.log


Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/connectionpool.py", line 799, in urlopen
    retries = retries.increment(
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/urllib3/util/retry.py", line 592, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /api/v1/admin/sys-info/?from=dtable_web (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f320740d580>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/seatable/seatable-server-latest/dtable-web/seahub/api2/endpoints/admin/utils.py", line 32, in get_dtable_server_info
    sys_info = dtable_server_api.get_sys_info()
  File "/opt/seatable/seatable-server-latest/dtable-web/seahub/dtable_apps/dtable_server_api.py", line 501, in get_sys_info
    response = requests.get(url, headers=admin_headers)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/opt/seatable/seatable-server-latest/dtable-web/thirdpart/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=5000): Max retries exceeded with url: /api/v1/admin/sys-info/?from=dtable_web (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f320740d580>: Failed to establish a new connection: [Errno 111] Connection refused'))
2025-02-05 12:11:37 [WARNING] django.request[241] - Not Found: /favicon.ico
2025-02-05 12:11:38 [WARNING] django.request[241] - Not Found: /favicon.ico
2025-02-05 12:11:38 [WARNING] django.request[241] - Not Found: /favicon.ico
2025-02-05 12:18:22 [WARNING] django.request[241] - Not Found: /api/v1/admin/sys-info/
2025-02-05 12:19:15 [WARNING] django.request[241] - Not Found: /workspace/4/dtables/Administrim 2024/
2025-02-05 12:25:03 [ERROR] seahub.api2.views[247] - scripts_running_count error: HTTPConnectionPool(host='python-scheduler', port=80): Max retries exceeded with url: /scripts-running-count/?username=8345ad0307a043a2b9560f66a4fc7bf2%40auth.local (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fde19ffb9a0>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
2025-02-05 12:31:57 [ERROR] seahub.api2.endpoints.admin.dtables[826] - HTTPConnectionPool(host='127.0.0.1', port=6000): Max retries exceeded with url: /add-export-task?username=8345ad0307a043a2b9560f66a4fc7bf2%40auth.local&table_name=Administrim+2023&repo_id=d9995dbc-cd23-4c30-9cd2-f6de9dc7f212&workspace_id=4&dtable_uuid=ee67013a-dc86-48c2-b498-27e61f96eb7f&ignore_asset=False (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fde19f2da90>: Failed to establish a new connection: [Errno 111] Connection refused'))
2025-02-05 12:31:57 [ERROR] django.request[241] - Internal Server Error: /api/v2.1/admin/dtables/ee67013a-dc86-48c2-b498-27e61f96eb7f/synchronous-export/export-dtable/
2025-02-05 12:35:27 [WARNING] django.request[241] - Not Found: /favicon.ico
2025-02-05 12:35:27 [WARNING] django.request[241] - Not Found: /favicon.ico

dtable_server_config.json

{
    "host": "mariadb",
    "user": "xxxxx",
    "password": "xxxxxx",
    "database": "xxxxxxx",
    "port": 3306,
    "private_key": "xxxxxxxxx",
    "redis_host": "redis",
    "redis_port": 6379,
    "redis_password": "",
    "dtable_web_service_url": "https://tables.domain.com/",
    "rows_api_hour_max": 600000,
    "rows_api_minute_max": 10000,
    "api_req_max": 60000,
    "api_base_day_max": 500000,
    "rows_api_max_limit": 1000000
}

I bumped up the rate limit settings just in case there was a rate limit somehow but it didn’t fix anything.

Is there any way to check if something got corrupted in the DB?

Let me know if there is anything else I can provide.

Thanks in advance!

Weirdly enough, a server reboot fixed it. I actually recreated the containers before rebooting and it didn’t fix it.

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.