[Rails Scaling] Add ability to easily have separate DB for engines

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[Rails Scaling] Add ability to easily have separate DB for engines

Kelly Stannard
Hello,

Summary

After watching the @eileencodes RailsConf keynote, I decided to resurrect an idea that Rails engines were the key to easy Rails scaling in larger companies since it promotes separation while being easier than SOA. Part of that separation is the need to have engines that can connect to their own database, promoting the portability of the engine.

Dan Manges articulates the gist of the engines only idea pretty well.
This is my original, functioning project from 2 years ago.
Current progress towards this goal.

Details of how an engine will work

Start with a separate, but otherwise normal, database.yml stored in the engine. The engine's namespaced ApplicationRecord will establish a connection using that database.yml. Table prefix is also dropped since it is in its own database.

I think there is probably a better, less verbose way to have runtime stuff happen, but just haven't figured it out yet.

In addition, we should expose new rake tasks, something like this from my original project.

Details of how the plugin generator will work

Right now I just added a `separate_database` flag so the full command would be:

be rails plugin --full --mountable --separate_database bar_service

Thanks for your time,

Kelly Wolf Stannard

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Rails Scaling] Add ability to easily have separate DB for engines

James Adam

It’s an interesting idea, but I’m not sure that a lot of people will need or even want to isolate engines into their own databases — in most cases, I’d expect people to want to link their application models to engine models (e.g. users to posts), which means they’d naturally sit in the same database.


I think you’ll need to make a very clear use-case for separating databases on a per-engine basis to get traction.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Rails Scaling] Add ability to easily have separate DB for engines

Kelly Stannard
Thank you for your response!

You are right in that single database is the easiest and most natural form of application. I think people reach for services way too early so don't expect an impassioned defense of SOA from me. That said, I think that multiple database is a feature that will have buy in for much the same reason SOA has buy in. Some people can't handle all their requests with a single database. For reference, this is Eileen's keynote:

https://youtu.be/8evXWvM4oXM?t=1130

Basically, think of this as SOA, but the applications are engines, and instead of communicating via HTTP, the applications communicate via method calls.

On Wednesday, June 20, 2018 at 11:03:19 AM UTC-4, James Adam wrote:

It’s an interesting idea, but I’m not sure that a lot of people will need or even want to isolate engines into their own databases — in most cases, I’d expect people to want to link their application models to engine models (e.g. users to posts), which means they’d naturally sit in the same database.


I think you’ll need to make a very clear use-case for separating databases on a per-engine basis to get traction.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: [Rails Scaling] Add ability to easily have separate DB for engines

Sachin Mudgal
I second Kelly's response. Even one of my current applications is having requirement of interacting with multiple database for different purposes.

The product(http://fanmojo.in) is itself on micro-service based architecture, where a few parts are developed as individual micro-services in Rails. Now, one Rails micro-service itself needs to interact with multiple databases for different purposes.

Say, Admin-Panel is micro-service in Rails, now it needs to interact in the following ways:
  1. MySQL for User-base type of data.
  2. DynamoDB/MongoDB for NoSQL type data.
  3. Redis for current state of Live/Volatile type of data.
So, a namespace based models could have easily separated out things.

Even while migrating our data via application interface(had to use application interface due to certain schema changes), we had do some little hack-type tricks to make it done(though it was easy stuff too).

On Wed, Jun 20, 2018 at 8:46 PM Kelly Stannard <[hidden email]> wrote:
Thank you for your response!

You are right in that single database is the easiest and most natural form of application. I think people reach for services way too early so don't expect an impassioned defense of SOA from me. That said, I think that multiple database is a feature that will have buy in for much the same reason SOA has buy in. Some people can't handle all their requests with a single database. For reference, this is Eileen's keynote:

https://youtu.be/8evXWvM4oXM?t=1130

Basically, think of this as SOA, but the applications are engines, and instead of communicating via HTTP, the applications communicate via method calls.

On Wednesday, June 20, 2018 at 11:03:19 AM UTC-4, James Adam wrote:

It’s an interesting idea, but I’m not sure that a lot of people will need or even want to isolate engines into their own databases — in most cases, I’d expect people to want to link their application models to engine models (e.g. users to posts), which means they’d naturally sit in the same database.


I think you’ll need to make a very clear use-case for separating databases on a per-engine basis to get traction.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/rubyonrails-core.
For more options, visit https://groups.google.com/d/optout.