Uploading and Importing Data
=============================

gtagora-connector supports three upload workflows:

1. **Simple upload** — directly upload files or directories into a folder.
2. **Advanced upload** — resumable, integrity-verified sessions.
3. **Custom import** — attach a JSON template to control how files are
   parsed into studies and series.

Simple upload
-------------

Upload individual files
~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

   from pathlib import Path

   folder = agora.get_folder(45)
   folder.upload([Path('/data/scan.raw'), Path('/data/scan.lab')])

Upload an entire directory
~~~~~~~~~~~~~~~~~~~~~~~~~~

The full sub-directory tree is replicated inside the target folder:

.. code-block:: python

   folder.upload([Path('/data/my_study/')])

Upload with related files
~~~~~~~~~~~~~~~~~~~~~~~~~

When Agora imports a raw file it creates a new Series. You can attach
additional files (e.g. a log file) to that Series by declaring *relations*:

.. code-block:: python

   files = [
       Path('/data/raw/scan.raw'),
       Path('/data/raw/scan.lab'),
       Path('/data/log/scan.log'),
   ]
   relations = {
       '/data/raw/scan.raw': ['/data/log/scan.log'],
   }
   folder.upload(files, relations=relations)

The same works when uploading a directory:

.. code-block:: python

   folder.upload([Path('/data/')], relations=relations)

Advanced upload (resumable)
----------------------------

The advanced upload creates a *session* that tracks progress in a local JSON
file. Interrupted uploads can be resumed from where they left off, and Agora
verifies each file's hash after transfer.

Create and start a session
~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

   from pathlib import Path

   files         = [Path('/data/scan.raw'), Path('/data/scan.lab')]
   progress_file = Path('/tmp/upload_progress.json')

   session = agora.create_upload_session(
       files,
       progress_file=progress_file,
       target_folder_id=45,
       verbose=True,
   )
   session.start()

Resume an interrupted upload
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

.. code-block:: python

   progress_file = Path('/tmp/upload_progress.json')
   session = agora.create_upload_session(progress_file=progress_file)
   session.start()

After the session completes, Agora waits for the import to finish and
verifies that every file was imported successfully.

Custom import via JSON template
--------------------------------

When Agora cannot automatically identify a file format (e.g. Philips PAR/REC),
you can supply a JSON import template that contains the patient, study and
series metadata.

.. code-block:: python

   import json
   from pathlib import Path

   # Connect and load an existing exam to attach new data to
   exam = agora.get_exam(37)

   file_paths = [
       Path('/data/2d_ffe.par'),
       Path('/data/2d_ffe.rec'),
   ]

   # Generate a template pre-filled with the exam's metadata
   import_json = agora.create_import_template(exam=exam, files=file_paths)

   # Customise the series metadata as needed
   import_json['ImportParameter']['Series']['Name'] = 'My New Series'
   import_json['ImportParameter']['Series']['AcquisitionNumber'] = 7

   # Save the template and pass it to the upload call
   json_file = Path('/tmp/import_template.json')
   json.dump(import_json, open(json_file, 'w'), indent=2)

   agora.upload(
       file_paths,
       json_import_file=json_file,
       target_folder_id=15,
       wait=True,
       verbose=True,
   )

After the upload you should see a new Series named ``My New Series`` inside
the existing study.