タケユー・ウェブ日報

Ruby on Rails や Flutter といったWeb・モバイルアプリ技術を武器にお客様のビジネス立ち上げを支援する、タケユー・ウェブ株式会社の技術ブログです。

AmazonLinux2にgcc6をインストール(標準のgcc7と共存させる)

まとめ

  • AmazonLinux2 標準の gcc は 7.x
  • gcc 6.x を使いたいが汚したくない
  • Environment modules で使いたいものだけ使う

事の起こり

事情によりHyper Estraierをインストールする必要があったのですが、通常通りに make すると QDBM が Segmentation fault して動きませんでした。

[root@ip-10-0-1-37 hyperestraier-1.4.13]# make check rm -rf casket casket- LD_LIBRARY_PATH=.:/lib:/usr/lib:/usr/local/lib:/root/lib:/usr/local/lib::/usr/local/lib ./estcmd create -tr -xl -attr '@uri' seq -attr '@title' str \ -attr '@author' str -attr '@mdate' num -attr '@size' num casket ./estcmd: INFO: status: name=casket dnum=0 wnum=0 fsiz=19924864 crnum=0 csiz=0 dknum=0 make: ** [check] Segmentation fault

[root@ip-10-0-1-37 qdbm-1.8.78]# make check (snip) rm -rf casket* LD_LIBRARY_PATH=.:/lib:/usr/lib:/usr/local/lib:/root/lib:/usr/local/lib ./odtest write casket 500 50 5000 name=casket dnum=500 wnum=50 pnum=5000 ibnum=-1 idnum=-1 cbnum=-1 csiz=-1

.make: *** [check] Segmentation fault

いろいろ情報をあさっていると、似た報告がありました。どうやら gcc 7 だと発生し、 gcc 6 なら発生しないようです。

estcmd built with gcc-7.2.0 caused segfault. I sent mails to the author, but I couldn't get a reply.

http://linuxplayers.g1.xrea.com/mozc-ut2.html

AmazonLInux2 の gcc

gcc version 7.3.1 20180712 (Red Hat 7.3.1-6) (GCC)

このgccは維持しながら、必要なときに gcc 6 を使えるようにしたい。

Environment modules

Environment modules が使えそうです。

参考 Environment modules事始め

Environment modules のインストール

$ sudo yum install environment-modules

gcc6 のインストール

yum install autogen

wget http://ftp.tsukuba.wide.ad.jp/software/gcc/releases/gcc-6.5.0/gcc-6.5.0.tar.gz -O /tmp/gcc-6.5.0.tar.gz tar xvf /tmp/gcc-6.5.0.tar.gz -C /usr/local/src cd /usr/local/src/gcc-6.5.0 ./contrib/download_prerequisites mkdir build && cd build ../configure --enable-languages=c,c++ --prefix=/usr/local/gcc-6.5.0 --disable-bootstrap --disable-multilib make && make check && make install

ソースディレクトリ内で configure しない理由 https://gcc.gnu.org/wiki/FAQ#configure

gcc6 を Environment modules で使うための設定

$ vi /etc/modulefiles/gcc-6.5.0
#%Module 1.0
#
#  gcc-6.5.0 module for use with 'environment-modules' package:
#

set GCChome /usr/local/gcc-6.5.0

prepend-path    PATH             $GCChome/bin
prepend-path    INCLUDE        $GCChome/include
prepend-path    LD_LIBRARY_PATH $GCChome/lib
prepend-path    MANPATH        $GCChome/man

使い方

モジュール一覧

$ module avail

----------------------- /usr/share/Modules/modulefiles ------------------------
dot         module-git  module-info modules     null        use.own

------------------------------ /etc/modulefiles -------------------------------
gcc-6.5.0

モジュールを使う

$ module add gcc-6.5.0
$ gcc --version
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/gcc-6.5.0/libexec/gcc/x86_64-pc-linux-gnu/6.5.0/lto-wrapper
Target: x86_64-pc-linux-gnu
Configured with: ../configure --enable-languages=c,c++ --prefix=/usr/local/gcc-6.5.0 --disable-bootstrap --disable-multilib
Thread model: posix
gcc version 6.5.0 (GCC)

使い終わったらアンロード

$ module unload gcc-6.5.0
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/libexec/gcc/x86_64-redhat-linux/7/lto-wrapper
Target: x86_64-redhat-linux
Configured with: ../configure --enable-bootstrap --enable-languages=c,c++,objc,obj-c++,fortran,ada,go,lto --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-shared --enable-threads=posix --enable-checking=release --enable-multilib --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-linker-build-id --with-gcc-major-version-only --with-linker-hash-style=gnu --enable-plugin --enable-initfini-array --with-isl --enable-libmpx --enable-libsanitizer --enable-gnu-indirect-function --enable-libcilkrts --enable-libatomic --enable-libquadmath --enable-libitm --with-tune=generic --with-arch_32=x86-64 --build=x86_64-redhat-linux
Thread model: posix
gcc version 7.3.1 20180712 (Red Hat 7.3.1-6) (GCC)

Rais.env.development? みたいな文字列がその値か?真偽値をとれるやつを使いたい

こういうのです。

Rails.env.production? # Rails.env == "production" と同じ

ActiveSupport::StringInquirer クラスを使います。

role = ActiveSupport::StringInquirer.new("admin")
role.admin? #=> true

ActiveSupport::StringInquirer

activesupport/lib/active_support/string_inquirer.rb

String のサブクラスで、respond_to_missing?method_missing を使っていますね。

メソッド名が ? で終わっていたら method_missing が呼ばれ、メソッド名の ? より前の部分文字列が自身と同じなら真を返すわけですね。

Active::Recordのトランザクション復習

user = User.find(2)
user.name = 'takeyuweb'
user.save!

User.transaction do         # A
  user.name = 'takeyuweb(trA)'
  user.save!
  
  User.transaction do       # B
    user.name = 'takeyuweb(trB)'
    user.save!
  end
  User.transaction do   # C
    user.name = 'takeyuweb(trC)'
    user.save!
    raise
  end
end

pp user.reload.name # => "takeyuweb"

user = User.find(2)
user.name = 'takeyuweb'
user.save!

User.transaction do         # A
  user.name = 'takeyuweb(trA)'
  user.save!
  
  User.transaction do       # B
    user.name = 'takeyuweb(trB)'
    user.save!
  end
  
  User.transaction(requires_new: true) do   # C
    user.name = 'takeyuweb(trC)'
    user.save!
    raise("突き抜けろ!")
  end
end
# RuntimeError: 突き抜けろ!

pp user.reload.name # => "takeyuweb"

--

user = User.find(2)
user.name = 'takeyuweb'
user.save!

User.transaction do         # A
  user.name = 'takeyuweb(trA)'
  user.save!
  
  User.transaction do       # B
    user.name = 'takeyuweb(trB)'
    user.save!
  end
  
  User.transaction(requires_new: true) do   # C
    user.name = 'takeyuweb(trC)'
    user.save!
    raise(ActiveRecord::Rollback)
  end
end
# CのActiveRecord::Rollbackは握りつぶされてそのまま処理を継続するので

pp user.reload.name # => "takeyuweb(trB)"

user = User.find(2)
user.name = 'takeyuweb'
user.save!

User.transaction do         # A
  User.transaction do       # B
    user.name = 'takeyuweb(trB)'
    user.save!
  end
  
  User.transaction(requires_new: true) do   # C
    user.name = 'takeyuweb(trC)'
    user.save!
    raise(ActiveRecord::Rollback)
  end
  
  user.name = 'takeyuweb(trA)'
  user.save!
end

pp user.reload.name # => "takeyuweb(trA)"

こうなる。

そのまま処理を継続してしまうとよくわかって使うこと

ruby aws sdk v2 SimpleWorkFlow(Aws::SWF::Client)メモ

swf = Aws::SWF::Client.new

失敗したExecutionを取り出す

WORKFLOW_DOMAIN = 'myworkflowdomain'

resp = swf.list_closed_workflow_executions(
         domain: WORKFLOW_DOMAIN,
         start_time_filter: {oldest_date: 24.hours.ago}, 
         close_status_filter: {status: 'TIMED_OUT'})
resp.execution_infos.each do |execution_info|
  pp execution_info
  # #<struct Aws::SWF::Types::WorkflowExecutionInfo
  #   execution=
  #    #<struct Aws::SWF::Types::WorkflowExecution
  #     workflow_id="fcac73cf-51c2-43f7-95cd-387a0bb4338b",
  #     run_id="22I3gtPVgL2GuGcBd9q1YvqH0D4aY66UbeQrJh+h7ExI8=">,
  #   workflow_type=
  #    #<struct Aws::SWF::Types::WorkflowType
  #     name="EncodeWorkflow.encode",
  #     version="1.20">,
  #   start_timestamp=2016-05-19 18:21:53 +0000,
  #   close_timestamp=2016-05-19 19:21:53 +0000,
  #   execution_status="CLOSED",
  #   close_status="TIMED_OUT",
  #   parent=nil,
  #   tag_list=[],
  #   cancel_requested=false>

  execution_event = swf.get_workflow_execution_history(
           domain: WORKFLOW_DOMAIN, 
           execution: execution_info.execution.to_h, 
           maximum_page_size: 1, 
           reverse_order: true).events.first
  pp execution_event
  # #<struct Aws::SWF::Types::HistoryEvent
  #  event_timestamp=2016-05-19 19:21:53 +0000,
  #  event_type="WorkflowExecutionTimedOut",
  #  event_id=183,
  #  workflow_execution_started_event_attributes=nil,
  #  workflow_execution_completed_event_attributes=nil,
  #  complete_workflow_execution_failed_event_attributes=nil,
  #  workflow_execution_failed_event_attributes=nil,
  #  fail_workflow_execution_failed_event_attributes=nil,
  #  workflow_execution_timed_out_event_attributes=
  #   #<struct Aws::SWF::Types::WorkflowExecutionTimedOutEventAttributes
  #    timeout_type="START_TO_CLOSE",
  #    child_policy="TERMINATE">,
  #  workflow_execution_canceled_event_attributes=nil,
  #  cancel_workflow_execution_failed_event_attributes=nil,
  #  workflow_execution_continued_as_new_event_attributes=nil,
  #  continue_as_new_workflow_execution_failed_event_attributes=nil,
  #  workflow_execution_terminated_event_attributes=nil,
  #  workflow_execution_cancel_requested_event_attributes=nil,
  #  decision_task_scheduled_event_attributes=nil,
  #  decision_task_started_event_attributes=nil,
  #  decision_task_completed_event_attributes=nil,
  #  decision_task_timed_out_event_attributes=nil,
  #  activity_task_scheduled_event_attributes=nil,
  #  activity_task_started_event_attributes=nil,
  #  activity_task_completed_event_attributes=nil,
  #  activity_task_failed_event_attributes=nil,
  #  activity_task_timed_out_event_attributes=nil,
  #  activity_task_canceled_event_attributes=nil,
  #  activity_task_cancel_requested_event_attributes=nil,
  #  workflow_execution_signaled_event_attributes=nil,
  #  marker_recorded_event_attributes=nil,
  #  record_marker_failed_event_attributes=nil,
  #  timer_started_event_attributes=nil,
  #  timer_fired_event_attributes=nil,
  #  timer_canceled_event_attributes=nil,
  #  start_child_workflow_execution_initiated_event_attributes=nil,
  #  child_workflow_execution_started_event_attributes=nil,
  #  child_workflow_execution_completed_event_attributes=nil,
  #  child_workflow_execution_failed_event_attributes=nil,
  #  child_workflow_execution_timed_out_event_attributes=nil,
  #  child_workflow_execution_canceled_event_attributes=nil,
  #  child_workflow_execution_terminated_event_attributes=nil,
  #  signal_external_workflow_execution_initiated_event_attributes=nil,
  #  external_workflow_execution_signaled_event_attributes=nil,
  #  signal_external_workflow_execution_failed_event_attributes=nil,
  #  external_workflow_execution_cancel_requested_event_attributes=nil,
  #  request_cancel_external_workflow_execution_initiated_event_attributes=nil,
  #  request_cancel_external_workflow_execution_failed_event_attributes=nil,
  #  schedule_activity_task_failed_event_attributes=nil,
  #  request_cancel_activity_task_failed_event_attributes=nil,
  #  start_timer_failed_event_attributes=nil,
  #  cancel_timer_failed_event_attributes=nil,
  #  start_child_workflow_execution_failed_event_attributes=nil,
  #  lambda_function_scheduled_event_attributes=nil,
  #  lambda_function_started_event_attributes=nil,
  #  lambda_function_completed_event_attributes=nil,
  #  lambda_function_failed_event_attributes=nil,
  #  lambda_function_timed_out_event_attributes=nil,
  #  schedule_lambda_function_failed_event_attributes=nil,
  #  start_lambda_function_failed_event_attributes=nil>
end

再実行

# 最初のイベントにワークフロー開始時の属性が入っているのでそれを使うことにした
execution_started_event_attributes = swf.get_workflow_execution_history(
           domain: WORKFLOW_DOMAIN, 
           execution: execution_info.execution.to_h, 
           maximum_page_size: 1, 
           reverse_order: false).events.first.workflow_execution_started_event_attributes

re_run_id = swf.start_workflow_execution({
  domain: WORKFLOW_DOMAIN,
  workflow_id: execution_info.execution.workflow_id,
  workflow_type: execution_info.workflow_type.to_h,
  task_list: execution_started_event_attributes.task_list,
  task_priority: execution_started_event_attributes.task_priority,
  input: execution_started_event_attributes.input,
  execution_start_to_close_timeout: execution_started_event_attributes.execution_start_to_close_timeout,
  tag_list: execution_started_event_attributes.tag_list,
  task_start_to_close_timeout: execution_started_event_attributes.task_start_to_close_timeout,
  child_policy: execution_started_event_attributes.child_policy,
  lambda_role: execution_started_event_attributes.lambda_role,
}).run_id

pp re_run_id # => "22K7jh/0kr9D4MGohM6aAESvZXkwuJg8WN3dN28silj1U="

OpsWorks (Chef11) で ERROR: cannot load such file -- /var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/files/lib/compat_resource/gemspec

いつも通りアプリのデプロイをしようとしたらエラー。

[2016-05-18T22:55:06+00:00] INFO: Loading cookbooks [apache2, apt, ark, build-essential, chef-sugar, chef_handler, compat_resource, curl, dependencies, deploy, gem_support, homebrew, mingw, mod_php5_apache2, mysql, newrelic, nginx, nodejs, opsworks, opsworks_agent_monit, opsworks_aws_flow_ruby, opsworks_bundler, opsworks_cleanup, opsworks_commons, opsworks_initial_setup, opsworks_java, opsworks_nodejs, opsworks_postgresql, opsworks_route53, opsworks_rubygems, opsworks_stack_state_sync, packages, passenger_apache2, php, python, rails, route53, ruby, s3_file, scm_helper, seven_zip, ssh_users, td-agent, test_suite, unicorn, windows, xml, yum, yum-epel]
 
================================================================================
Recipe Compile Error in /var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/libraries/autoload.rb
================================================================================
 
 
LoadError
---------
cannot load such file -- /var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/files/lib/compat_resource/gemspec
 
 
Cookbook Trace:
---------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/libraries/autoload.rb:15:in `require_relative'
/var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/libraries/autoload.rb:15:in `<top (required)>'
 
 
Relevant File Content:
----------------------
/var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/libraries/autoload.rb:
 
8:    cookbook_version = $1
9:  
10:    if CompatResource::VERSION != cookbook_version
11:      raise "compat_resource gem version #{CompatResource::VERSION} was loaded as a gem before compat_resource cookbook version #{cookbook_version} was loaded. To remedy this, either update the cookbook to the gem version, update the gem to the cookbook version, or uninstall / stop loading the gem so early."
12:    end
13:  else
14:    # The gem is not already activated, so activate the cookbook.
15>>   require_relative '../files/lib/compat_resource/gemspec'
16:    CompatResource::GEMSPEC.activate
17:  end
18:  
19:  require 'compat_resource'
20:  
 
 
[2016-05-18T22:55:06+00:00] ERROR: Running exception handlers
[2016-05-18T22:55:06+00:00] ERROR: Exception handlers complete
[2016-05-18T22:55:06+00:00] FATAL: Stacktrace dumped to /var/lib/aws/opsworks/cache.stage2/chef-stacktrace.out
[2016-05-18T22:55:06+00:00] ERROR: cannot load such file -- /var/lib/aws/opsworks/cache.stage2/cookbooks/compat_resource/files/lib/compat_resource/gemspec
[2016-05-18T22:55:06+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

解決までの経過まとめ

発覚

  • compat_resource という Chef12用のクックブックを使おうとしているみたい。ほかのコミュニティクックブックの依存がかわった?

Chef12に

Chef11のまましのぐには

  • 動いていたときと動かなくなったときの依存クックブックを比較(OpsWorksのログをみる)
  • Installing build-essential (4.0.0)これぽい
    • 依存クックブックの依存クックブック
    • 4.0.0からChef12以降専用になった
  • Berksfileで動いていたときのバージョン(3.2.0)に固定
cookbook 'build-essential', '3.2.0'

再デプロイ

古いバージョン固定なので、いずれ腐ってきてどうにもならなくなったときつらそうだが、当面はこれで・・・