Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

A way to merge databases (or multiple instances) into one -- or a way to export results from an instance and import them elsewhere #480

Open
dmsimard opened this issue Feb 10, 2023 · 2 comments
Labels
built-in server/offline client Related to "ara-manage runserver" or the offline API client database Related to the database backends or models

Comments

@dmsimard
Copy link
Contributor

I don't personally have a use case for this right now but it's something I've been asked about a lot so I am at least creating an issue for it 😅

What is the idea ?

At a high level, the idea is to be able to run ansible playbooks with ara in different places, each saving to their own database (sqlite or not) and then a way to export them to another instance to aggregate the results after the fact.

The distributed sqlite backend kind of does that but it keeps the individual databases "sharded": it does not merge them such that all the results are viewable from a single web reporting interface or API server.

distributed_sqlite

We can't just blindly merge databases together -- there's primary keys (playbook IDs), foreign keys, etc. but a quick search about doing this with django projects suggests it is possible in different ways.

An approach that might work regardless of database backend could be to crawl everything available over the API and POST everything back to the another one. I expect this would be slow at scale and not without challenges like figuring out whether the data we're importing already exists so we're not importing it a second time 😬

Anyway, this is an issue where we might consider what are the possible options and then figure something out.

If you need this, have ideas, suggestions or would like to contribute, feel free to comment here.

@dmsimard dmsimard added database Related to the database backends or models built-in server/offline client Related to "ara-manage runserver" or the offline API client labels Feb 10, 2023
@hille721
Copy link
Contributor

What exactly are use cases for that? Can not really imagine where this could help me. I would also guess that this is a thing which is really specific from use case to use case.
One thing which comes to my mind is that if I have air-gapped networks with no possibility to send the data to one Ara instance. But if it's really air gapped I can't merge it afterwards too. Thus also there I can not see an advantage...

@julien-gagnon
Copy link

julien-gagnon commented Mar 9, 2023

For example, say I want to migrate use of ara to a new environment where there's already an ara API server up and running, and being used. Ideally, I'd like to keep the ability to browse logs in one place, old and new
.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
built-in server/offline client Related to "ara-manage runserver" or the offline API client database Related to the database backends or models
Projects
None yet
Development

No branches or pull requests

3 participants