Usage

Getting help

If you need some information about a command you can always pass the --help flag to get some information.

Generate a config file

First you can generate a template config file:

$ fits2db init

Note

you can also pass a path to the command to choose where the config file should be generated and what the name of the file should be:

$ fits2db init <folder_path>

this generates a file config.yml under your given path

$ fits2db init <example/path/config_test.yaml>

this gernerates the config file with the name config_test.yaml.

Make your changes

In the config.yml file you now can change the variables needed.

Fill in the database credentials:

database:
  type: mysql
  host: localhost
  user: user
  password: password
  db_name: test_db
  port: 3306

and add some paths for your fits files

fits_files:
  paths:
    - path/to_your_file/2021-07-07_L1a.fits
    - path_to_your_folder

# Delete rows from above listed files from tables which are not listed below. By default False
delete_rows_from_missing_tables: True

tables:
    - name: HOUSEKEEPING
      date_column: timestamp # This column will be interpreted as a datetime variable
    - name: IRRADIANCE # If no table name given it will use the orignal name
      date_column: irradiance_timeutc

Note

if a folder is given all fits files under this folder will be taken recursively for upload.

Note

if the date column is not 'timestamp', a copy of the date column called 'timestamp' is created. This is due to backward compatibility reasons.

Check if the right files are taken

You can check if you get the right fits files with

$ fits2db files <path_to_config_file>

this command shows all files it will consider uploading in your terminal and at the end shows the number of files.

Note

If you don't add an path the cli looks for the config file in the same folder as you are currently in.

Inspect available Tables

If you want to see what tables are available in your fits files you can run

$ fits2db tables <path_to_config_file> 

this will get you a summary of all tables

Tip

If you want to see if the tables are available in all files just run the sam command with the matrix flag

$ fits2db tables <path_to_config_file> -m

this will show the reult in the terminal. If you want to have and excel or csv use

$ fits2db tables <path_to_config_file> -m --excel --filename path/your_filename.xlsx

Build db

Now upload the data into our data base we use the build command

$ fits2db build <path_to_config_file> 

this will upload all the fits tables into your data base and create the meta tables to keep track on changes of the files When running thei build command the user is prompted to confirm the action to remove all existing data from the configured database. If the user denies, the build command is aborted.

Warning

If you rerun the build command it acts as an reset. It will drop the tables and reupload all data to have a fresh start. This is only recommend to use when you lost track of some changes in the data you have done manually and you are not sure you corrupted the data.

Update db

Once builded and you get new files or changes you can update the database. This command will check if there a new files in your defnied folders and upload them to the db. If the timestamp of your file changed to a newer date. Like when you changed a file it will also update this file to the newer version. This way the fits files and the db stay in sync. To update just run

$ fits2db update <path_to_config_file> 

Note

If you want to add new tables from a already updated file, you can use the -f flag to force update all files specified in the config. This can for example be used to add additional tables from a already uploaded file to the database.

Remove files from tables

With the remove_rows_from_missing_tables option, one can remove entries form columns. For example if a upladed file has entries in Table a and b and the update command is excecuted with only the table a configured and remove_rows_from_missing_tables set to True, then the rows from all configured files from table b will be removed.