Python Pipeline

goran.kuga · July 15, 2024, 5:26pm

Your Setup:

Virtualized environment VMware
SeaTable Edition Enterprise
SeaTable Version 5.0.7

Describe the Problem/Error/Question:

Please describe your issue as precise as possible. If helpful, you can also provide:

Manual pages you used SeaTable-Server - SeaTable Admin Manual and Separate Python Pipeline - SeaTable Admin Manual

Dear Support,

I’ve started with building new SeaTable environment including separate SeaTable server and Python Pipeline. I followed instructions for both of those precisely but when attempting to execute script from SeaTable it does not work showing: error: Error: Internal Server Error.

Could you please point me to correct direction at least to get some logs ?

For Python Pipeline I adjusted .env to COMPOSE_FILE=‘caddy.yml,python-pipeline.yml’

I did have to adjust python-pipeline.yml file in order for it to start working.

I adjusted (removed this part): networks: - backend-seatable-net

If I don’t remove it, container will not start, showing an error.

Could you please assist me further to get this working ?

I did update with URL and AUTH key on SeaTable side in dtable_web_server.

goran.kuga · July 15, 2024, 5:27pm

SeaTable Server Docker:

goran.kuga · July 15, 2024, 5:28pm

SeaTable Python Pipeline:

goran.kuga · July 15, 2024, 5:28pm

Message I get when executing a script from SeaTable:

goran.kuga · July 15, 2024, 5:29pm

As I mentioned that I had to remove network from .yml file. If I don’t remove it this is what happens:

Or if anyone can provide precise instructions on how to build separate Python Pipeline.

cdb · July 15, 2024, 8:21pm

Hey Goran,
can you please post the output of these commands?

docker logs python-starter
docker logs python-scheduler

Does SeaTable and Python-Pipeline have separate but public available domains?

I assume that the problem is, that python-scheduler does not find the database for initialization.
Did you define DB_HOST in the .env? Is the MariaDB reachable over the network?

goran.kuga · July 16, 2024, 11:42am

Hey,

Thank you so much for your reply.

They both have same domain but different DNS, example: seatable.company.com and python.company.com. DNS is resolvable between them and they can communicate without any restrictions.

As I looked at the logs I would say that is correct, it is showing issues with DB. I did not place DB_HOST in the .env as I did not see it in documentation anywhere. I presume that this has to be setup on Python Pipeline to point to SeaTable server were DB is ?

PS: Logs from docker:

Thanks !

Kind regards,
Goran

goran.kuga · July 16, 2024, 11:42am

goran.kuga · July 16, 2024, 11:46am

What would be a good example of how DB_HOST is configured in .env ?

goran.kuga · July 16, 2024, 12:00pm

I did adjustments on seatable-server.yml and exposed MariaDB.

Tested from Python Pipeline server and through telnet its reachable on port 3306.

How do I configure now Python Pipeline to connect to it? Should I place DNS of my SeaTable server in DB_HOST that I place in .env file or ? How does this work ?

Thanks !

goran.kuga · July 16, 2024, 12:11pm

Hey,

I figured it out. So in .env on Python Pipeline server, variable named SEATABLE_MYSQL_HOST= should be placed not DB_HOST.

After that its running, at least logs are showing it from Python Pipeline server:

I tried again executing script from SeaTable but again getting error: Error: Internal Server Error.

goran.kuga · July 16, 2024, 12:39pm

On my SeaTable server I adjust .env with following, placed for scheduler URL on http as per seatable-server.yml file that is in example. AUTH is generated and placed on both servers same in .env.

PYTHON_SCHEDULER_URL=
PYTHON_SCHEDULER_AUTH_TOKEN=

cdb · July 16, 2024, 1:48pm

I am sorry, it seems that my description how to deploy python pipeline on a separate server is not covering all necessary requirements. Unfortunately, I don’t have time today to have a look at this.
Give me some time and I will have a look.
Probably this description of the python pipeline workflow helps you to check the different components part of python pipeline:

goran.kuga · July 17, 2024, 1:20pm

Dear,

Any feedback ?

Thanks

Kind regards,
Goran

goran.kuga · July 22, 2024, 7:09am

Dear,

I tried standalone version and its also not working, though the error is different. Could you please review your documentation on how to build this?

With previous version documentation was great, this is really…

Thanks !

Kind regards,
Goran

goran.kuga · July 31, 2024, 11:45am

Dear,

Any feedback ?

Thanks

Kind regards,
Goran

cdb · July 31, 2024, 2:24pm

I updated the documentation to install a standalone python-pipeline on a separate host: Separate Python Pipeline - SeaTable Admin Manual.

goran.kuga · July 31, 2024, 3:08pm

Dear,

I looked at documentation but cannot find anything over there that was not already mentioned here in a ticket.

Is this proven to be working, you tested it out on your own? As it still does not work for me, and its completely the same.

I’m still getting this issue:
I tried again executing script from SeaTable but again getting error: Error: Internal Server Error.

Thanks !

Kind regards,
Goran

cdb · July 31, 2024, 7:24pm

Dear @goran.kuga,

I can confirm that the new documentation is working. I updated and tested the documentation by myself today.

Did you really take the necessary time to understand what I changed?

new python-pipeline-standalone.yml file with MariaDB container included
new env variable to expose the python-scheduler to a public URL via caddy

Here are the concrete changes from Github:

My recommendation: delete all containers and .yml files from your python-pipeline host and follow the documentation. Then it will work.

goran.kuga · August 1, 2024, 10:36am

Dear,

I did everything from scratch with new files pulled from repository, adjusted them properly. And everything is indeed healthy under docker, no issues when looking docker logs for each container, but still it does not work.

Whenever I try Python script from SeaTable it shows error: Error: Internal Server Error.

If I do inspect on the page to see errors it shows following: (I hidden start of URL).

Failed to load resource: the server responded with a status of 500 ()
https://MY_SEA_TABLE_URL/api/v2.1/dtable/dd0c1599-34b5-44f8-8d89-2b33ef169e3f/run-script/TiGw.py/

Could you please let me know where I can find the log of what happens when I execute Python script from SeaTable, where I can see that it actually tries or connect to Python Pipeline?

I’m using simple script:
print(“Hello World”)

PS: Servers are reachable between, I did curl to confirm. When I do a curl toward Python Pipeline URL or open that URL in browser of my jump server I see following, not sure if I should see anything different.

404 Not Found

Not Found

The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.

Thanks !

Kind regards,
Goran