ActiveRecord's connection pool

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

ActiveRecord's connection pool

Janko Marohnić
I was studying the difference between Sequel's and ActiveRecord's connection pool, and I was curious why doesn't ActiveRecord automatically release connections back into the pool after it was used for a query? Currently it stays checked out and assigned to the original thread, so other threads cannot use it, even when the original thread died.

To illustrate, the following script executes 10 queries in parallel in 10 threads, but because the pool size is 5, half of the threads will fail because they couldn't acquire a connection:

require "active_record"

ActiveRecord::Base.establish_connection(
  adapter
:  "sqlite3",
  database
: "database.sqlite3",
)

threads
= []

10.times do
  threads
<< Thread.new do
   
ActiveRecord::Base.connection.execute "SELECT 1"
 
end
end

threads
.each(&:join)

ActiveRecord::ConnectionTimeoutError: could not obtain a connection from the pool within 5.000 seconds (waited 5.003 seconds); all pooled connections were in use

Compare that to Sequel, where everything goes just fine, since it releases connections back into the pool as soon as the query finished:

require "sequel"

DB
= Sequel.sqlite("database.sqlite3", max_connections: 5)

threads
= []

10.times do
  threads
<< Thread.new do
    DB
.run "SELECT 1"
 
end
end

threads
.each(&:join)

ActiveRecord works around this limitation by requiring users to call `ActiveRecord::Base.clear_active_connections!` at the appropriate place (end of request in a web app). In Rails this is handled automatically for you (though at a surprising place – in `ActiveRecord::QueryCache.complete`), but you need to remember to do it when using ActiveRecord in other web frameworks. In Rails this started as a specialized Rack middleware called ConnectionManagement, but has since been moved into ActionDispatch::Executor middleware.

My question is whether the current connection pool behaviour is intentional (maybe there is some advantage I'm not seeing?), or it's just that no one volunteered to fix it yet?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/1b81bedd-2cf5-44f4-a079-7b1808fd6345%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveRecord's connection pool

botp
On 7/2/19, Janko Marohnić <[hidden email]> wrote:

>
> 10.times do
>   threads << Thread.new do
>     ActiveRecord::Base.connection.execute "SELECT 1"

try also remove_connection

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAAwHHQhefaNNg_Pw3ywUYQqKXQBWCjET-24emFnX%2B6%2BO2xQ0RQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: ActiveRecord's connection pool

Janko Marohnić
ActiveRecord::Base.remove_connection disconnects all connections to the database. ActiveRecord might reconnect on subsequent queries, but this is not what I want, I want to reuse already open connections for performance.

To be clear, I know how to make the above code work (I can call ActiveRecord::Base.clear_active_connections!, or ActiveRecord::Base.connection_pool.with_connection with a block), what I’m challenging is that ActiveRecord shouldn’t require that you manually release connections back into the pool (like Sequel doesn’t).
On 2 Jul 2019, 17:48 +0200, botp <[hidden email]>, wrote:
On 7/2/19, Janko Marohnić <[hidden email]> wrote:


10.times do
threads << Thread.new do
ActiveRecord::Base.connection.execute "SELECT 1"

try also remove_connection

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/CAAwHHQhefaNNg_Pw3ywUYQqKXQBWCjET-24emFnX%2B6%2BO2xQ0RQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Talk" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-talk/adbbafac-50be-4976-9aae-d32980d088ef%40Spark.
For more options, visit https://groups.google.com/d/optout.