module Sidekiq
SdNotify
is a pure-Ruby implementation of sd_notify(3). It can be used to notify systemd about state changes. Methods of this package are no-op on non-systemd systems (eg. Darwin).
The API maps closely to the original implementation of sd_notify(3), therefore be sure to check the official man pages prior to using SdNotify
.
@see www.freedesktop.org/software/systemd/man/sd_notify.html
Sidekiq's systemd integration allows Sidekiq
to inform systemd:
1. when it has successfully started 2. when it is starting shutdown 3. periodically for a liveness check with a watchdog thread
Constants
- DEFAULTS
- DEFAULT_WORKER_OPTIONS
- FAKE_INFO
- Job
Sidekiq::Job
is a new alias forSidekiq::Worker
as ofSidekiq
6.3.0. Use `include Sidekiq::Job` rather than `include Sidekiq::Worker`.The term “worker” is too generic and overly confusing, used in several different contexts meaning different things. Many people call a
Sidekiq
process a “worker”. Some people call the thread that executes jobs a “worker”. This change bringsSidekiq
closer to ActiveJob where your job classes extend ApplicationJob.- LICENSE
- NAME
- VERSION
- Workers
The
WorkSet
stores the work being done by thisSidekiq
cluster. It tracks the process and thread working on each job.WARNING WARNING WARNING
This is live data that can change every millisecond. If you call size => 5 and then expect each to be called 5 times, you're going to have a bad time.
works = Sidekiq::WorkSet.new works.size => 2 works.each do |process_id, thread_id, work| # process_id is a unique identifier per Sidekiq process # thread_id is a unique identifier per thread # work is a Hash which looks like: # { 'queue' => name, 'run_at' => timestamp, 'payload' => job_hash } # run_at is an epoch Integer. end
Public Class Methods
How frequently Redis should be checked by a random Sidekiq
process for scheduled and retriable jobs. Each individual process will take turns by waiting some multiple of this value.
See sidekiq/scheduled.rb for an in-depth explanation of this value
# File lib/sidekiq.rb, line 226 def self.average_scheduled_poll_interval=(interval) options[:average_scheduled_poll_interval] = interval end
# File lib/sidekiq.rb, line 143 def self.client_middleware @client_chain ||= Middleware::Chain.new yield @client_chain if block_given? @client_chain end
Configuration for Sidekiq
client, use like:
Sidekiq.configure_client do |config| config.redis = { :namespace => 'myapp', :size => 1, :url => 'redis://myhost:8877/0' } end
# File lib/sidekiq.rb, line 86 def self.configure_client yield self unless server? end
Configuration for Sidekiq
server, use like:
Sidekiq.configure_server do |config| config.redis = { :namespace => 'myapp', :size => 25, :url => 'redis://myhost:8877/0' } config.server_middleware do |chain| chain.add MyServerHook end end
# File lib/sidekiq.rb, line 76 def self.configure_server yield self if server? end
Death handlers are called when all retries for a job have been exhausted and the job dies. It's the notification to your application that this job will not succeed without manual intervention.
Sidekiq.configure_server
do |config|
config.death_handlers << ->(job, ex) do end
end
# File lib/sidekiq.rb, line 177 def self.death_handlers options[:death_handlers] end
# File lib/sidekiq.rb, line 155 def self.default_server_middleware Middleware::Chain.new end
# File lib/sidekiq.rb, line 164 def self.default_worker_options defined?(@default_worker_options) ? @default_worker_options : DEFAULT_WORKER_OPTIONS end
# File lib/sidekiq.rb, line 159 def self.default_worker_options=(hash) # stringify @default_worker_options = default_worker_options.merge(hash.transform_keys(&:to_s)) end
# File lib/sidekiq.rb, line 185 def self.dump_json(object) JSON.generate(object) end
Register a proc to handle any error which occurs within the Sidekiq
process.
Sidekiq.configure_server do |config| config.error_handlers << proc {|ex,ctx_hash| MyErrorService.notify(ex, ctx_hash) } end
The default error handler logs errors to Sidekiq.logger
.
# File lib/sidekiq.rb, line 237 def self.error_handlers options[:error_handlers] end
# File lib/sidekiq.rb, line 181 def self.load_json(string) JSON.parse(string) end
# File lib/sidekiq.rb, line 189 def self.log_formatter @log_formatter ||= if ENV["DYNO"] Sidekiq::Logger::Formatters::WithoutTimestamp.new else Sidekiq::Logger::Formatters::Pretty.new end end
# File lib/sidekiq.rb, line 197 def self.log_formatter=(log_formatter) @log_formatter = log_formatter logger.formatter = log_formatter end
# File lib/sidekiq.rb, line 202 def self.logger @logger ||= Sidekiq::Logger.new($stdout, level: Logger::INFO) end
# File lib/sidekiq.rb, line 206 def self.logger=(logger) if logger.nil? self.logger.level = Logger::FATAL return self.logger end logger.extend(Sidekiq::LoggingUtils) @logger = logger end
Register a block to run at a point in the Sidekiq
lifecycle. :startup, :quiet or :shutdown are valid events.
Sidekiq.configure_server do |config| config.on(:shutdown) do puts "Goodbye cruel world!" end end
# File lib/sidekiq.rb, line 249 def self.on(event, &block) raise ArgumentError, "Symbols only please: #{event}" unless event.is_a?(Symbol) raise ArgumentError, "Invalid event name: #{event}" unless options[:lifecycle_events].key?(event) options[:lifecycle_events][event] << block end
# File lib/sidekiq.rb, line 59 def self.options @options ||= DEFAULTS.dup end
# File lib/sidekiq.rb, line 63 def self.options=(opts) @options = opts end
# File lib/sidekiq.rb, line 217 def self.pro? defined?(Sidekiq::Pro) end
# File lib/sidekiq.rb, line 94 def self.redis raise ArgumentError, "requires a block" unless block_given? redis_pool.with do |conn| retryable = true begin yield conn rescue Redis::BaseError => ex # 2550 Failover can cause the server to become a replica, need # to disconnect and reopen the socket to get back to the primary. # 4495 Use the same logic if we have a "Not enough replicas" error from the primary # 4985 Use the same logic when a blocking command is force-unblocked if retryable && ex.message =~ /READONLY|NOREPLICAS|UNBLOCKED/ conn.disconnect! retryable = false retry end raise end end end
# File lib/sidekiq.rb, line 135 def self.redis=(hash) @redis = if hash.is_a?(ConnectionPool) hash else Sidekiq::RedisConnection.create(hash) end end
# File lib/sidekiq.rb, line 115 def self.redis_info redis do |conn| # admin commands can't go through redis-namespace starting # in redis-namespace 2.0 if conn.respond_to?(:namespace) conn.redis.info else conn.info end rescue Redis::CommandError => ex # 2850 return fake version when INFO command has (probably) been renamed raise unless /unknown command/.match?(ex.message) FAKE_INFO end end
# File lib/sidekiq.rb, line 131 def self.redis_pool @redis ||= Sidekiq::RedisConnection.create end
# File lib/sidekiq.rb, line 90 def self.server? defined?(Sidekiq::CLI) end
# File lib/sidekiq.rb, line 149 def self.server_middleware @server_chain ||= default_server_middleware yield @server_chain if block_given? @server_chain end
# File lib/sidekiq/systemd.rb, line 8 def self.start_watchdog usec = Integer(ENV["WATCHDOG_USEC"]) return Sidekiq.logger.error("systemd Watchdog too fast: " + usec) if usec < 1_000_000 sec_f = usec / 1_000_000.0 # "It is recommended that a daemon sends a keep-alive notification message # to the service manager every half of the time returned here." ping_f = sec_f / 2 Sidekiq.logger.info "Pinging systemd watchdog every #{ping_f.round(1)} sec" Thread.new do loop do sleep ping_f Sidekiq::SdNotify.watchdog end end end
# File lib/sidekiq.rb, line 55 def self.❨╯°□°❩╯︵┻━┻ puts "Calm down, yo." end