[Feature][ActiveRecord] blocklist : ignored_columns :: allowlist : feature_idea

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

[Feature][ActiveRecord] blocklist : ignored_columns :: allowlist : feature_idea

Dan Ott
ActiveRecord has the ActiveRecord::Base.ignored_columns = %w(some columns to ignore) method to blocklist columns that ActiveRecord loads from the database. Spelunking the ActiveRecord source, I found no corresponding method that acts as an allowlist for columns that will be loaded.I found myself reaching for an allowlist as I started to take advantage of Rails 6's multi-database support. In my context, data from another database that used to be wrapped in a REST endpoint can now be accessed directly. When making this direct access, I want to ignore every attribute except the few I am specifically interested in. Because we’re in a multiple-app environment, we don’t necessarily have control over columns being added or dropped from the secondary database during runtime. We don’t want our application to go down because another team removed an experimental column from their database.To solve this problem with ignored_columns, we had to enumerate all the existing columns.
class Dogs < AnimalsBase
 self.ignored_columns = %w(some list of columns that can never be exhaustive because new columns could be added all the time)
end
This suffers from the problem of new columns could be added at any time. The list of ignored columns has to be constantly tended to ensure we’re robust against a runtime error of columns going away.What I wanted to reach for was something like
class Dogs < AnimalsBase
 self.allowed_columns = %w(id and only the exact columns needed)
end
With this approach, we’re robust against the other database adding and removing columns. We still run the risk of encountering runtime errors if one of our exactly-requested columns goes away. I can’t come up with a way to mitigate that risk aside from improving communications between teams.I’m not solid on the name allowed_columns but it was the first thing I reached for.For the time being, I implemented this with a monkey patch to load_schema!
ActiveRecord::Base.concerning "AllowedColumns" do
 included do
   self.allowed_columns = [].freeze
 end

 module ClassMethods
   def allowed_columns
     if defined?(@allowed_columns)
       @allowed_columns
     else
       superclass.allowed_columns
     end
   end

   def allowed_columns=(columns)
     @allowed_columns = columns.map(&:to_s)
   end

   # copied from from https://github.com/rails/rails/blob/2dea8c29c794bec564a2e69ad715d25024e93932/activerecord/lib/active_record/model_schema.rb#L484-L497
   def load_schema!
     unless table_name
       raise ActiveRecord::TableNotSpecified, "#{self} has no table configured. Set one with #{self}.table_name="
     end
     @columns_hash = connection.schema_cache.columns_hash(table_name).except(*ignored_columns)
     @columns_hash = @columns_hash.slice(*allowed_columns) if allowed_columns.present? # This is the additional line
     @columns_hash.each do |name, column|
       define_attribute(
         name,
         connection.lookup_cast_type_from_column(column),
         default: column.default,
         user_provided_default: false
       )
     end
   end
 end
end
Is this a feature that others would find useful now that we’re working with first-class multi-database support?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-core/0580e7d8-845c-4300-8bec-03968710ad25%40googlegroups.com.
Reply | Threaded
Open this post in threaded view
|

Re: [Feature][ActiveRecord] blocklist : ignored_columns :: allowlist : feature_idea

Gabriel Sobrinho
I'm not from the core but in my experience the chances are better opening a PR with the feature, specially if the implementation is simple.

I can even review the code, I've made a few PRs before with the implementation of the ignored columns and I'm familiar with the code.

On Thursday, August 29, 2019 at 2:29:15 PM UTC-3, Dan Ott wrote:
ActiveRecord has the ActiveRecord::Base.ignored_columns = %w(some columns to ignore) method to blocklist columns that ActiveRecord loads from the database. Spelunking the ActiveRecord source, I found no corresponding method that acts as an allowlist for columns that will be loaded.I found myself reaching for an allowlist as I started to take advantage of Rails 6's multi-database support. In my context, data from another database that used to be wrapped in a REST endpoint can now be accessed directly. When making this direct access, I want to ignore every attribute except the few I am specifically interested in. Because we’re in a multiple-app environment, we don’t necessarily have control over columns being added or dropped from the secondary database during runtime. We don’t want our application to go down because another team removed an experimental column from their database.To solve this problem with ignored_columns, we had to enumerate all the existing columns.
class Dogs < AnimalsBase
 self.ignored_columns = %w(some list of columns that can never be exhaustive because new columns could be added all the time)
end
This suffers from the problem of new columns could be added at any time. The list of ignored columns has to be constantly tended to ensure we’re robust against a runtime error of columns going away.What I wanted to reach for was something like
class Dogs < AnimalsBase
 self.allowed_columns = %w(id and only the exact columns needed)
end
With this approach, we’re robust against the other database adding and removing columns. We still run the risk of encountering runtime errors if one of our exactly-requested columns goes away. I can’t come up with a way to mitigate that risk aside from improving communications between teams.I’m not solid on the name allowed_columns but it was the first thing I reached for.For the time being, I implemented this with a monkey patch to load_schema!
ActiveRecord::Base.concerning "AllowedColumns" do
 included do
   self.allowed_columns = [].freeze
 end

 module ClassMethods
   def allowed_columns
     if defined?(@allowed_columns)
       @allowed_columns
     else
       superclass.allowed_columns
     end
   end

   def allowed_columns=(columns)
     @allowed_columns = columns.map(&:to_s)
   end

   # copied from from <a href="https://github.com/rails/rails/blob/2dea8c29c794bec564a2e69ad715d25024e93932/activerecord/lib/active_record/model_schema.rb#L484-L497" rel="nofollow" style="color:inherit" target="_blank" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frails%2Frails%2Fblob%2F2dea8c29c794bec564a2e69ad715d25024e93932%2Factiverecord%2Flib%2Factive_record%2Fmodel_schema.rb%23L484-L497\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHz7ZXJ1WTqHOqg6ImH99QKnKdJmw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Frails%2Frails%2Fblob%2F2dea8c29c794bec564a2e69ad715d25024e93932%2Factiverecord%2Flib%2Factive_record%2Fmodel_schema.rb%23L484-L497\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHz7ZXJ1WTqHOqg6ImH99QKnKdJmw&#39;;return true;">https://github.com/rails/rails/blob/2dea8c29c794bec564a2e69ad715d25024e93932/activerecord/lib/active_record/model_schema.rb#L484-L497
   def load_schema!
     unless table_name
       raise ActiveRecord::TableNotSpecified, "#{self} has no table configured. Set one with #{self}.table_name="
     end
     @columns_hash = connection.schema_cache.columns_hash(table_name).except(*ignored_columns)
     @columns_hash = @columns_hash.slice(*allowed_columns) if allowed_columns.present? # This is the additional line
     @columns_hash.each do |name, column|
       define_attribute(
         name,
         connection.lookup_cast_type_from_column(column),
         default: column.default,
         user_provided_default: false
       )
     end
   end
 end
end
Is this a feature that others would find useful now that we’re working with first-class multi-database support?

--
You received this message because you are subscribed to the Google Groups "Ruby on Rails: Core" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/rubyonrails-core/7f881ecd-88ea-4892-8ac1-d8c10a8e274b%40googlegroups.com.