前言
本文主要介绍了关于PostgreSQL中Slony-I同步复制部署的相关内容，分享出来供大家参考学习，下面话不多说了，来一起看看详细的介绍吧
本次测试环境
IP 10.189.102.118 10.189.100.195 10.189.100.226 PGHOME /usr/local/pgsql /usr/local/pgsql /usr/local/pgsql Role Origin,Providers Subscribers,Providers Subscribers Database region_il region_il region_il Table
il_longlive.entity
il_longlive.entity_result
il_longlive.result
il_longlive.result_logbook
il_longlive.entity
il_longlive.entity_result
il_longlive.result
il_longlive.result_logbook
il_longlive.entity
il_longlive.entity_result
il_longlive.result
il_longlive.result_logbook
Sequencesil_longlive.entity_entity_id_seq
il_longlive.entity_result_id_seq
il_longlive.result_id_seq
il_longlive.result_logbook_record_id_seq
il_longlive.entity_entity_id_seq
il_longlive.entity_result_id_seq
il_longlive.result_id_seq
il_longlive.result_logbook_record_id_seq
il_longlive.entity_entity_id_seq
il_longlive.entity_result_id_seq
il_longlive.result_id_seq
il_longlive.result_logbook_record_id_seq
Slony-I一般要求表有主键,或者唯一键,Slony-I会自动识别主键,如果表没有主键,需要手工指定唯一键,如果没有指定唯一键,则不能复制,下图为Slony-I架构图.
1. 查看主库表信息
region_il=# \d
List of relations
Schema | Name | Type | Owner
-------------+------------------------------+----------+----------
il_longlive | entity | table | postgres
il_longlive | entity_entity_id_seq | sequence | postgres
il_longlive | entity_result | table | postgres
il_longlive | entity_result_id_seq | sequence | postgres
il_longlive | result | table | postgres
il_longlive | result_id_seq | sequence | postgres
il_longlive | result_logbook | table | postgres
il_longlive | result_logbook_record_id_seq | sequence | postgres
il_longlive | result_vendor_schema | table | postgres
il_longlive | result_vendor_schema_id_seq | sequence | postgres
il_longlive | snapshot | table | postgres
il_longlive | snapshot_id_seq | sequence | postgres
(12 rows)
region_il=# \d entity
Table "il_longlive.entity"
Column | Type | Modifiers
--------------+------------------------+------------------------------------------------------------
entity_id | integer | not null default nextval('entity_entity_id_seq'::regclass)
country_code | character varying(3) |
state_code | character varying(50) |
name | character varying(128) |
islive | boolean |
Indexes:
"entity_pkey" PRIMARY KEY, btree (entity_id)
region_il=# select count(*) from entity;
count
---------
(1 row)
2. 在所有节点安装Slony-I
$ wget http://www.slony.info/downloads/2.2/source/slony1-2.2.4.tar.bz2 $ tar -jxf slony1-2.2.4.tar.bz2 $ cd slony1-2.2.4 $ ./configure --prefix=/usr/local/pgsql --with-pgconfigdir=/usr/local/pgsql/bin --with-perltools $ make $ make install
3. 在所有节点创建Slony-I使用的专用用户
$ psql -c "CREATE USER slonyadmin WITH SUPERUSER REPLICATION LOGIN PASSWORD '888888';"
4. 在所有节点设置允许Slony-I用户远程登录
$ grep slonyadmin /usr/local/pgsql/data/pg_hba.conf host all slonyadmin 0.0.0.0/0 md5
5. 在所有节点的对应数据库创建plpgsql语言
$ createlang plpgsql -d region_il
6. 在主节点创建Slony-I配置文件,并复制到所有其他节点
$ cat /usr/local/pgsql/etc/slon_tools.conf
if ($ENV{"SLONYNODES"}) {
require $ENV{"SLONYNODES"};
} else {
$CLUSTER_NAME = 'region_il';
$PIDFILE_DIR = '/usr/local/pgsql/data';
$LOGDIR = '/usr/local/pgsql/data/pg_log';
$MASTERNODE = 1;
$DEBUGLEVEL = 0;
$START_WATCHDOG = 1;
$SLEEP_TIME = 60;
$WATCHDOG_VERBOSE = 0;
add_node(node => 1,
host => '10.189.102.118',
dbname => 'region_il',
port => 5432,
user => 'slonyadmin',
password => '888888',);
add_node(node => 2,
parent => 1,
host => '10.189.100.195',
dbname => 'region_il',
port => 5432,
user => 'slonyadmin',
password => '888888',);
add_node(node => 3,
parent => 2,
host => '10.189.100.226',
dbname => 'region_il',
port => 5432,
user => 'slonyadmin',
password => '888888',);
}
$SLONY_SETS = {
"il_longlive_schema" => {
"set_id" => 1,
"table_id" => 101,
"sequence_id" => 101,
"pkeyedtables" => [
'il_longlive.entity',
'il_longlive.entity_result',
'il_longlive.result',
'il_longlive.result_logbook',
],
"keyedtables" => {},
"sequences" => ['il_longlive.entity_entity_id_seq',
'il_longlive.entity_result_id_seq',
'il_longlive.result_id_seq',
'il_longlive.result_logbook_record_id_seq',
],
},
};
if ($ENV{"SLONYSET"}) {
require $ENV{"SLONYSET"};
}
# Please do not add or change anything below this point.
1;
$ scp /usr/local/pgsql/etc/slon_tools.conf postgres@10.189.100.195:/usr/local/pgsql/etc/
$ scp /usr/local/pgsql/etc/slon_tools.conf postgres@10.189.100.226:/usr/local/pgsql/etc/
7. 将需要复制的表结构从主节点导出并导入到所有辅节点中
$ pg_dump -U slonyadmin -s -t il_longlive.entity -t il_longlive.entity_result -t il_longlive.result -t il_longlive.result_logbook region_il | psql -h 10.189.100.195 -U slonyadmin region_il $ pg_dump -U slonyadmin -s -t il_longlive.entity -t il_longlive.entity_result -t il_longlive.result -t il_longlive.result_logbook region_il | psql -h 10.189.100.226 -U slonyadmin region_il
8. 在主节点初始化Slony-I集群
$ /usr/local/pgsql/bin/slonik_init_cluster # INIT CLUSTER cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; init cluster (id = 1, comment = 'Node 1 - region_il@10.189.102.118'); # STORE NODE store node (id = 2, event node = 1, comment = 'Node 2 - region_il@10.189.100.195'); store node (id = 3, event node = 1, comment = 'Node 3 - region_il@10.189.100.226'); echo 'Set up replication nodes'; # STORE PATH echo 'Next: configure paths for each node/origin'; store path (server = 1, client = 2, conninfo = 'host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 1, client = 3, conninfo = 'host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 2, client = 1, conninfo = 'host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 2, client = 3, conninfo = 'host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 3, client = 1, conninfo = 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 3, client = 2, conninfo = 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'); echo 'Replication nodes prepared'; echo 'Please start a slon replication daemon for each node'; $ /usr/local/pgsql/bin/slonik_init_cluster | slonik <stdin>:7: Possible unsupported PostgreSQL version (90505) 9.5, defaulting to 8.4 support <stdin>:10: Possible unsupported PostgreSQL version (90505) 9.5, defaulting to 8.4 support <stdin>:11: Possible unsupported PostgreSQL version (90505) 9.5, defaulting to 8.4 support <stdin>:12: Set up replication nodes <stdin>:15: Next: configure paths for each node/origin <stdin>:22: Replication nodes prepared <stdin>:23: Please start a slon replication daemon for each node
9. 在所有节点启动slon进程
在主节点启动slon进程
$ slon_start 1 Invoke slon for node 1 - /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node1.pid -s 1000 -d0 region_il 'host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888' > /usr/local/pgsql/data/pg_log/node1/region_il-2017-06-14.log 2>&1 & Slon successfully started for cluster region_il, node node1 PID [2454] Start the watchdog process as well... $ slon_status 1 Slon is running for the 'region_il' cluster on node1. $ ps -ef | grep slon postgres 2454 1 0 22:54 pts/1 00:00:00 /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node1.pid -s 1000 -d0 region_il host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888 postgres 2455 2454 0 22:54 pts/1 00:00:00 /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node1.pid -s 1000 -d0 region_il host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888 postgres 2459 24883 0 22:54 "htmlcode">$ slon_start 2 Invoke slon for node 2 - /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node2.pid -s 1000 -d0 region_il 'host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888' > /usr/local/pgsql/data/pg_log/node2/region_il-2017-06-14.log 2>&1 & Slon successfully started for cluster region_il, node node2 PID [15535] Start the watchdog process as well... $ slon_status 2 Slon is running for the 'region_il' cluster on node2. $ ps -ef | grep slon postgres 15515 12843 0 22:54 "htmlcode">$ slon_start 3 Invoke slon for node 3 - /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node3.pid -s 1000 -d0 region_il 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888' > /usr/local/pgsql/data/pg_log/node3/region_il-2017-06-14.log 2>&1 & Slon successfully started for cluster region_il, node node3 PID [19212] Start the watchdog process as well... $ slon_status 3 Slon is running for the 'region_il' cluster on node3. $ ps -ef | grep slon postgres 6033 1 0 Jun02 "color: #ff0000">10. 在主节点创建复制集$ /usr/local/pgsql/bin/slonik_create_set 1 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; # TABLE ADD KEY # CREATE SET create set (id = 1, origin = 1, comment = 'Set 1 (il_longlive_schema) for region_il'); # SET ADD TABLE echo 'Subscription set 1 (il_longlive_schema) created'; echo 'Adding tables to the subscription set'; set add table (set id = 1, origin = 1, id = 101, full qualified name = 'il_longlive.entity', comment = 'Table il_longlive.entity with primary key'); echo 'Add primary keyed table il_longlive.entity'; set add table (set id = 1, origin = 1, id = 102, full qualified name = 'il_longlive.entity_result', comment = 'Table il_longlive.entity_result with primary key'); echo 'Add primary keyed table il_longlive.entity_result'; set add table (set id = 1, origin = 1, id = 103, full qualified name = 'il_longlive.result', comment = 'Table il_longlive.result with primary key'); echo 'Add primary keyed table il_longlive.result'; set add table (set id = 1, origin = 1, id = 104, full qualified name = 'il_longlive.result_logbook', comment = 'Table il_longlive.result_logbook with primary key'); echo 'Add primary keyed table il_longlive.result_logbook'; # SET ADD SEQUENCE echo 'Adding sequences to the subscription set'; set add sequence (set id = 1, origin = 1, id = 101, full qualified name = 'il_longlive.entity_entity_id_seq', comment = 'Sequence il_longlive.entity_entity_id_seq'); echo 'Add sequence il_longlive.entity_entity_id_seq'; set add sequence (set id = 1, origin = 1, id = 102, full qualified name = 'il_longlive.entity_result_id_seq', comment = 'Sequence il_longlive.entity_result_id_seq'); echo 'Add sequence il_longlive.entity_result_id_seq'; set add sequence (set id = 1, origin = 1, id = 103, full qualified name = 'il_longlive.result_id_seq', comment = 'Sequence il_longlive.result_id_seq'); echo 'Add sequence il_longlive.result_id_seq'; set add sequence (set id = 1, origin = 1, id = 104, full qualified name = 'il_longlive.result_logbook_record_id_seq', comment = 'Sequence il_longlive.result_logbook_record_id_seq'); echo 'Add sequence il_longlive.result_logbook_record_id_seq'; echo 'All tables added'; $ /usr/local/pgsql/bin/slonik_create_set 1 | slonik <stdin>:12: Subscription set 1 (il_longlive_schema) created <stdin>:13: Adding tables to the subscription set <stdin>:17: Add primary keyed table il_longlive.entity <stdin>:21: Add primary keyed table il_longlive.entity_result <stdin>:25: Add primary keyed table il_longlive.result <stdin>:29: Add primary keyed table il_longlive.result_logbook <stdin>:32: Adding sequences to the subscription set <stdin>:36: Add sequence il_longlive.entity_entity_id_seq <stdin>:40: Add sequence il_longlive.entity_result_id_seq <stdin>:44: Add sequence il_longlive.result_id_seq <stdin>:48: Add sequence il_longlive.result_logbook_record_id_seq <stdin>:49: All tables added11. 在主节点创建复制订阅者
$ slonik_subscribe_set 1 2 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; subscribe set (id = 1, provider = 1, receiver = 2, forward = yes); echo 'Subscribed nodes to set 1'; $ slonik_subscribe_set 1 2 | slonik <stdin>:7: Subscribed nodes to set 1 $ slonik_subscribe_set 1 3 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; subscribe set (id = 1, provider = 2, receiver = 3, forward = yes); echo 'Subscribed nodes to set 1'; $ slonik_subscribe_set 1 3 | slonik <stdin>:7: Subscribed nodes to set 112.查看各节点Slony-I日志信息
主节点查看Slony-I日志信息
$ cat region_il-2017-06-14.log 2017-06-14 22:54:42 PDT CONFIG main: slon version 2.2.4 starting up 2017-06-14 22:54:42 PDT INFO slon: watchdog process started 2017-06-14 22:54:42 PDT CONFIG slon: watchdog ready - pid = 2454 2017-06-14 22:54:42 PDT CONFIG slon: worker process created - pid = 2455 2017-06-14 22:54:42 PDT CONFIG main: Integer option vac_frequency = 3 2017-06-14 22:54:42 PDT CONFIG main: Integer option log_level = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option sync_interval = 1000 2017-06-14 22:54:42 PDT CONFIG main: Integer option sync_interval_timeout = 10000 2017-06-14 22:54:42 PDT CONFIG main: Integer option sync_group_maxsize = 20 2017-06-14 22:54:42 PDT CONFIG main: Integer option syslog = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option quit_sync_provider = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option remote_listen_timeout = 300 2017-06-14 22:54:42 PDT CONFIG main: Integer option monitor_interval = 500 2017-06-14 22:54:42 PDT CONFIG main: Integer option explain_interval = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option tcp_keepalive_idle = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option tcp_keepalive_interval = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option tcp_keepalive_count = 0 2017-06-14 22:54:42 PDT CONFIG main: Integer option apply_cache_size = 100 2017-06-14 22:54:42 PDT CONFIG main: Boolean option log_pid = 0 2017-06-14 22:54:42 PDT CONFIG main: Boolean option log_timestamp = 1 2017-06-14 22:54:42 PDT CONFIG main: Boolean option tcp_keepalive = 1 2017-06-14 22:54:42 PDT CONFIG main: Boolean option monitor_threads = 1 2017-06-14 22:54:42 PDT CONFIG main: Real option real_placeholder = 0.000000 2017-06-14 22:54:42 PDT CONFIG main: String option cluster_name = region_il 2017-06-14 22:54:42 PDT CONFIG main: String option conn_info = host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888 2017-06-14 22:54:42 PDT CONFIG main: String option pid_file = /usr/local/pgsql/data/region_il_node1.pid 2017-06-14 22:54:42 PDT CONFIG main: String option log_timestamp_format = %Y-%m-%d %H:%M:%S %Z 2017-06-14 22:54:42 PDT CONFIG main: String option archive_dir = [NULL] 2017-06-14 22:54:42 PDT CONFIG main: String option sql_on_connection = [NULL] 2017-06-14 22:54:42 PDT CONFIG main: String option lag_interval = [NULL] 2017-06-14 22:54:42 PDT CONFIG main: String option command_on_logarchive = [NULL] 2017-06-14 22:54:42 PDT CONFIG main: String option syslog_facility = LOCAL0 2017-06-14 22:54:42 PDT CONFIG main: String option syslog_ident = slon 2017-06-14 22:54:42 PDT CONFIG main: String option cleanup_interval = 10 minutes 2017-06-14 22:54:42 PDT CONFIG main: local node id = 1 2017-06-14 22:54:42 PDT INFO main: main process started 2017-06-14 22:54:42 PDT CONFIG main: launching sched_start_mainloop 2017-06-14 22:54:42 PDT CONFIG main: loading current cluster configuration 2017-06-14 22:54:42 PDT CONFIG storeNode: no_id=2 no_comment='Node 2 - region_il@10.189.100.195' 2017-06-14 22:54:42 PDT CONFIG storeNode: no_id=3 no_comment='Node 3 - region_il@10.189.100.226' 2017-06-14 22:54:42 PDT CONFIG storePath: pa_server=2 pa_client=1 pa_conninfo="host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 22:54:42 PDT CONFIG storePath: pa_server=3 pa_client=1 pa_conninfo="host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG main: last local event sequence = 5000000007 2017-06-14 22:54:42 PDT CONFIG main: configuration complete - starting threads 2017-06-14 22:54:42 PDT INFO localListenThread: thread starts 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG enableNode: no_id=2 2017-06-14 22:54:42 PDT CONFIG enableNode: no_id=3 2017-06-14 22:54:42 PDT INFO main: running scheduler mainloop 2017-06-14 22:54:42 PDT INFO remoteWorkerThread_3: thread starts 2017-06-14 22:54:42 PDT INFO remoteListenThread_3: thread starts 2017-06-14 22:54:42 PDT CONFIG cleanupThread: thread starts 2017-06-14 22:54:42 PDT INFO syncThread: thread starts 2017-06-14 22:54:42 PDT INFO monitorThread: thread starts 2017-06-14 22:54:42 PDT INFO remoteListenThread_2: thread starts 2017-06-14 22:54:42 PDT INFO remoteWorkerThread_2: thread starts 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG cleanupThread: bias = 60 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 22:54:42 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 22:54:42 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:54:42 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 22:57:50 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 22:57:50 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:50 PDT INFO remoteWorkerThread_2: SYNC 5000000003 done in 0.014 seconds 2017-06-14 22:57:58 PDT INFO remoteWorkerThread_2: SYNC 5000000004 done in 0.003 seconds 2017-06-14 22:58:11 PDT INFO remoteWorkerThread_2: SYNC 5000000005 done in 0.002 seconds 2017-06-14 22:58:19 PDT INFO remoteWorkerThread_2: SYNC 5000000006 done in 0.002 seconds 2017-06-14 22:58:27 PDT INFO remoteWorkerThread_2: SYNC 5000000007 done in 0.003 seconds 2017-06-14 22:58:35 PDT INFO remoteWorkerThread_2: SYNC 5000000008 done in 0.002 seconds 2017-06-14 22:58:43 PDT INFO remoteWorkerThread_2: SYNC 5000000009 done in 0.002 seconds 2017-06-14 22:59:01 PDT INFO remoteWorkerThread_2: SYNC 5000000010 done in 0.003 seconds 2017-06-14 22:59:09 PDT INFO remoteWorkerThread_2: SYNC 5000000011 done in 0.002 seconds 2017-06-14 22:59:17 PDT INFO remoteWorkerThread_2: SYNC 5000000012 done in 0.002 seconds 2017-06-14 22:59:25 PDT INFO remoteWorkerThread_2: SYNC 5000000013 done in 0.002 seconds 2017-06-14 22:59:38 PDT INFO remoteWorkerThread_2: SYNC 5000000014 done in 0.003 seconds 2017-06-14 22:59:51 PDT INFO remoteWorkerThread_2: SYNC 5000000015 done in 0.002 seconds 2017-06-14 22:59:54 PDT INFO remoteWorkerThread_2: SYNC 5000000016 done in 0.002 seconds 2017-06-14 23:00:05 PDT INFO remoteWorkerThread_2: SYNC 5000000017 done in 0.002 seconds 2017-06-14 23:00:19 PDT INFO remoteWorkerThread_2: SYNC 5000000018 done in 0.003 seconds 2017-06-14 23:00:32 PDT INFO remoteWorkerThread_2: SYNC 5000000019 done in 0.003 seconds 2017-06-14 23:00:35 PDT INFO remoteWorkerThread_2: SYNC 5000000020 done in 0.003 seconds 2017-06-14 23:00:43 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:00:43 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:43 PDT INFO remoteWorkerThread_3: SYNC 5000000003 done in 0.045 seconds 2017-06-14 23:00:47 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:47 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:47 PDT INFO remoteWorkerThread_2: SYNC 5000000021 done in 0.014 seconds 2017-06-14 23:00:55 PDT INFO remoteWorkerThread_3: SYNC 5000000004 done in 0.002 seconds 2017-06-14 23:00:55 PDT INFO remoteWorkerThread_2: SYNC 5000000022 done in 0.005 seconds 2017-06-14 23:01:08 PDT INFO remoteWorkerThread_3: SYNC 5000000005 done in 0.002 seconds 2017-06-14 23:01:08 PDT INFO remoteWorkerThread_2: SYNC 5000000023 done in 0.005 seconds 2017-06-14 23:01:16 PDT INFO remoteWorkerThread_3: SYNC 5000000006 done in 0.002 seconds 2017-06-14 23:01:20 PDT INFO remoteWorkerThread_3: SYNC 5000000007 done in 0.002 seconds 2017-06-14 23:01:20 PDT INFO remoteWorkerThread_2: SYNC 5000000024 done in 0.005 seconds 2017-06-14 23:01:24 PDT INFO remoteWorkerThread_2: SYNC 5000000025 done in 0.005 seconds 2017-06-14 23:01:38 PDT INFO remoteWorkerThread_3: SYNC 5000000008 done in 0.003 seconds 2017-06-14 23:01:38 PDT INFO remoteWorkerThread_2: SYNC 5000000026 done in 0.005 seconds 2017-06-14 23:01:41 PDT INFO remoteWorkerThread_3: SYNC 5000000009 done in 0.002 seconds 2017-06-14 23:01:44 PDT INFO remoteWorkerThread_2: SYNC 5000000027 done in 0.004 seconds 2017-06-14 23:01:52 PDT INFO remoteWorkerThread_3: SYNC 5000000010 done in 0.003 seconds 2017-06-14 23:01:55 PDT INFO remoteWorkerThread_2: SYNC 5000000028 done in 0.005 seconds 2017-06-14 23:02:08 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:02:08 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:02:08 PDT INFO remoteWorkerThread_2: SYNC 5000000029 done in 0.012 seconds 2017-06-14 23:02:08 PDT INFO remoteWorkerThread_3: SYNC 5000000011 done in 0.016 seconds 2017-06-14 23:02:11 PDT INFO remoteWorkerThread_3: SYNC 5000000012 done in 0.004 seconds 2017-06-14 23:02:19 PDT INFO remoteWorkerThread_2: SYNC 5000000030 done in 0.005 seconds 2017-06-14 23:02:22 PDT INFO remoteWorkerThread_3: SYNC 5000000013 done in 0.005 seconds 2017-06-14 23:02:25 PDT INFO remoteWorkerThread_2: SYNC 5000000031 done in 0.004 seconds 2017-06-14 23:02:38 PDT INFO remoteWorkerThread_2: SYNC 5000000032 done in 0.004 seconds 2017-06-14 23:02:38 PDT INFO remoteWorkerThread_3: SYNC 5000000014 done in 0.005 seconds 2017-06-14 23:02:46 PDT INFO remoteWorkerThread_3: SYNC 5000000015 done in 0.005 seconds 2017-06-14 23:02:50 PDT INFO remoteWorkerThread_3: SYNC 5000000016 done in 0.004 seconds 2017-06-14 23:02:50 PDT INFO remoteWorkerThread_2: SYNC 5000000033 done in 0.005 seconds 2017-06-14 23:02:54 PDT INFO remoteWorkerThread_2: SYNC 5000000034 done in 0.004 seconds 2017-06-14 23:03:08 PDT INFO remoteWorkerThread_3: SYNC 5000000017 done in 0.004 seconds 2017-06-14 23:03:08 PDT INFO remoteWorkerThread_2: SYNC 5000000035 done in 0.005 seconds 2017-06-14 23:03:11 PDT CONFIG storeSet: set_id=1 set_origin=1 set_comment='Set 1 (il_longlive_schema) for region_il' 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_3: SYNC 5000000018 done in 0.005 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_2: SYNC 5000000036 done in 0.005 seconds 2017-06-14 23:03:21 PDT INFO remoteWorkerThread_3: SYNC 5000000019 done in 0.004 seconds 2017-06-14 23:03:29 PDT INFO remoteWorkerThread_2: SYNC 5000000037 done in 0.016 seconds 2017-06-14 23:03:32 PDT INFO remoteWorkerThread_3: SYNC 5000000020 done in 0.012 seconds 2017-06-14 23:03:35 PDT INFO remoteWorkerThread_2: SYNC 5000000038 done in 0.004 seconds 2017-06-14 23:03:48 PDT INFO remoteWorkerThread_2: SYNC 5000000039 done in 0.004 seconds 2017-06-14 23:03:48 PDT INFO remoteWorkerThread_3: SYNC 5000000021 done in 0.005 seconds 2017-06-14 23:03:56 PDT INFO remoteWorkerThread_3: SYNC 5000000022 done in 0.005 seconds 2017-06-14 23:03:57 PDT INFO remoteWorkerThread_2: SYNC 5000000040 done in 0.005 seconds 2017-06-14 23:04:01 PDT INFO remoteWorkerThread_3: SYNC 5000000023 done in 0.005 seconds 2017-06-14 23:04:04 PDT INFO remoteWorkerThread_2: SYNC 5000000041 done in 0.005 seconds 2017-06-14 23:04:17 PDT INFO remoteWorkerThread_2: SYNC 5000000042 done in 0.005 seconds 2017-06-14 23:04:17 PDT INFO remoteWorkerThread_3: SYNC 5000000024 done in 0.005 seconds 2017-06-14 23:04:20 PDT INFO remoteWorkerThread_3: SYNC 5000000025 done in 0.004 seconds 2017-06-14 23:04:23 PDT INFO remoteWorkerThread_2: SYNC 5000000043 done in 0.008 seconds 2017-06-14 23:04:31 PDT INFO remoteWorkerThread_3: SYNC 5000000026 done in 0.005 seconds 2017-06-14 23:04:34 PDT INFO remoteWorkerThread_2: SYNC 5000000044 done in 0.004 seconds 2017-06-14 23:04:42 PDT INFO remoteWorkerThread_3: SYNC 5000000027 done in 0.005 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2498 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2499 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2500 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2526 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: Logswitch to sl_log_2 initiated CONTEXT: SQL statement "SELECT "_region_il".logswitch_start()" PL/pgSQL function _region_il.cleanupevent(interval) line 97 at PERFORM 2017-06-14 23:04:43 PDT INFO cleanupThread: 0.011 seconds for cleanupEvent() 2017-06-14 23:04:45 PDT INFO remoteWorkerThread_2: SYNC 5000000045 done in 0.004 seconds 2017-06-14 23:04:53 PDT INFO remoteWorkerThread_2: SYNC 5000000046 done in 0.005 seconds 2017-06-14 23:04:53 PDT INFO remoteWorkerThread_3: SYNC 5000000028 done in 0.005 seconds 2017-06-14 23:05:06 PDT INFO remoteWorkerThread_3: SYNC 5000000029 done in 0.005 seconds 2017-06-14 23:05:06 PDT INFO remoteWorkerThread_2: SYNC 5000000047 done in 0.005 seconds 2017-06-14 23:05:19 PDT INFO remoteWorkerThread_2: SYNC 5000000048 done in 0.004 seconds 2017-06-14 23:05:19 PDT INFO remoteWorkerThread_3: SYNC 5000000030 done in 0.012 seconds 2017-06-14 23:05:22 PDT INFO remoteWorkerThread_3: SYNC 5000000031 done in 0.004 seconds 2017-06-14 23:05:25 PDT INFO remoteWorkerThread_2: SYNC 5000000049 done in 0.005 seconds 2017-06-14 23:05:38 PDT INFO remoteWorkerThread_2: SYNC 5000000050 done in 0.004 seconds 2017-06-14 23:05:38 PDT INFO remoteWorkerThread_3: SYNC 5000000032 done in 0.005 seconds 2017-06-14 23:05:41 PDT INFO remoteWorkerThread_3: SYNC 5000000033 done in 0.005 seconds 2017-06-14 23:05:45 PDT INFO remoteWorkerThread_2: SYNC 5000000051 done in 0.005 seconds 2017-06-14 23:05:53 PDT INFO remoteWorkerThread_3: SYNC 5000000034 done in 0.004 seconds 2017-06-14 23:05:53 PDT INFO remoteWorkerThread_2: SYNC 5000000052 done in 0.005 seconds 2017-06-14 23:06:06 PDT INFO remoteWorkerThread_3: SYNC 5000000035 done in 0.005 seconds 2017-06-14 23:06:06 PDT INFO remoteWorkerThread_2: SYNC 5000000053 done in 0.005 seconds 2017-06-14 23:06:14 PDT INFO remoteWorkerThread_2: SYNC 5000000054 done in 0.005 seconds 2017-06-14 23:06:14 PDT INFO remoteWorkerThread_3: SYNC 5000000036 done in 0.006 seconds 2017-06-14 23:06:22 PDT INFO remoteWorkerThread_3: SYNC 5000000037 done in 0.005 seconds 2017-06-14 23:06:25 PDT INFO remoteWorkerThread_2: SYNC 5000000055 done in 0.004 seconds 2017-06-14 23:06:30 PDT INFO remoteWorkerThread_3: SYNC 5000000038 done in 0.005 seconds 2017-06-14 23:06:36 PDT INFO remoteWorkerThread_2: SYNC 5000000056 done in 0.005 seconds 2017-06-14 23:06:40 PDT INFO remoteWorkerThread_3: SYNC 5000000039 done in 0.005 seconds 2017-06-14 23:06:48 PDT INFO remoteWorkerThread_2: SYNC 5000000057 done in 0.007 seconds 2017-06-14 23:06:56 PDT INFO remoteWorkerThread_3: SYNC 5000000040 done in 0.004 seconds 2017-06-14 23:06:56 PDT INFO remoteWorkerThread_2: SYNC 5000000058 done in 0.005 seconds 2017-06-14 23:07:09 PDT INFO remoteWorkerThread_3: SYNC 5000000041 done in 0.004 seconds 2017-06-14 23:07:09 PDT INFO remoteWorkerThread_2: SYNC 5000000059 done in 0.005 seconds 2017-06-14 23:07:10 PDT INFO remoteWorkerThread_3: SYNC 5000000042 done in 0.004 seconds 2017-06-14 23:07:19 PDT INFO remoteWorkerThread_2: SYNC 5000000060 done in 0.005 seconds 2017-06-14 23:07:23 PDT INFO remoteWorkerThread_3: SYNC 5000000043 done in 0.015 seconds 2017-06-14 23:07:23 PDT INFO remoteWorkerThread_2: SYNC 5000000061 done in 0.017 seconds 2017-06-14 23:07:31 PDT INFO remoteWorkerThread_3: SYNC 5000000044 done in 0.004 seconds 2017-06-14 23:07:39 PDT INFO remoteWorkerThread_2: SYNC 5000000062 done in 0.013 seconds 2017-06-14 23:07:42 PDT INFO remoteWorkerThread_3: SYNC 5000000045 done in 0.006 seconds 2017-06-14 23:07:43 PDT INFO remoteWorkerThread_2: SYNC 5000000063 done in 0.005 seconds 2017-06-14 23:07:51 PDT INFO remoteWorkerThread_3: SYNC 5000000046 done in 0.008 seconds 2017-06-14 23:07:56 PDT INFO remoteWorkerThread_2: SYNC 5000000064 done in 0.005 seconds 2017-06-14 23:08:02 PDT INFO remoteWorkerThread_3: SYNC 5000000047 done in 0.022 seconds 2017-06-14 23:08:07 PDT INFO remoteWorkerThread_2: SYNC 5000000065 done in 0.005 seconds 2017-06-14 23:08:13 PDT INFO remoteWorkerThread_3: SYNC 5000000048 done in 0.008 seconds 2017-06-14 23:08:18 PDT INFO remoteWorkerThread_2: SYNC 5000000066 done in 0.011 seconds 2017-06-14 23:08:24 PDT INFO remoteWorkerThread_3: SYNC 5000000049 done in 0.004 seconds 2017-06-14 23:08:24 PDT INFO remoteWorkerThread_2: SYNC 5000000067 done in 0.017 seconds 2017-06-14 23:08:35 PDT INFO remoteWorkerThread_3: SYNC 5000000050 done in 0.005 seconds 2017-06-14 23:08:39 PDT INFO remoteWorkerThread_2: SYNC 5000000068 done in 0.005 seconds 2017-06-14 23:08:40 PDT INFO remoteWorkerThread_3: SYNC 5000000051 done in 0.004 seconds 2017-06-14 23:08:49 PDT INFO remoteWorkerThread_2: SYNC 5000000069 done in 0.005 seconds 2017-06-14 23:08:53 PDT INFO remoteWorkerThread_2: SYNC 5000000070 done in 0.005 seconds 2017-06-14 23:08:54 PDT INFO remoteWorkerThread_3: SYNC 5000000052 done in 0.005 seconds 2017-06-14 23:09:02 PDT INFO remoteWorkerThread_3: SYNC 5000000053 done in 0.006 seconds 2017-06-14 23:09:07 PDT INFO remoteWorkerThread_2: SYNC 5000000071 done in 0.005 seconds 2017-06-14 23:09:13 PDT INFO remoteWorkerThread_3: SYNC 5000000054 done in 0.005 seconds 2017-06-14 23:09:18 PDT INFO remoteWorkerThread_2: SYNC 5000000072 done in 0.005 seconds 2017-06-14 23:09:24 PDT INFO remoteWorkerThread_3: SYNC 5000000055 done in 0.005 seconds 2017-06-14 23:09:29 PDT INFO remoteWorkerThread_2: SYNC 5000000073 done in 0.005 seconds 2017-06-14 23:09:33 PDT INFO remoteWorkerThread_2: SYNC 5000000074 done in 0.005 seconds 2017-06-14 23:09:34 PDT INFO remoteWorkerThread_3: SYNC 5000000056 done in 0.005 seconds 2017-06-14 23:09:48 PDT INFO remoteWorkerThread_3: SYNC 5000000057 done in 0.004 seconds 2017-06-14 23:09:48 PDT INFO remoteWorkerThread_2: SYNC 5000000075 done in 0.005 seconds 2017-06-14 23:09:52 PDT INFO remoteWorkerThread_3: SYNC 5000000058 done in 0.005 seconds 2017-06-14 23:09:55 PDT INFO remoteWorkerThread_2: SYNC 5000000076 done in 0.004 seconds 2017-06-14 23:10:00 PDT INFO remoteWorkerThread_3: SYNC 5000000059 done in 0.005 seconds 2017-06-14 23:10:06 PDT INFO remoteWorkerThread_2: SYNC 5000000077 done in 0.005 seconds 2017-06-14 23:10:11 PDT INFO remoteWorkerThread_3: SYNC 5000000060 done in 0.005 seconds 2017-06-14 23:10:17 PDT INFO remoteWorkerThread_2: SYNC 5000000078 done in 0.005 seconds 2017-06-14 23:10:22 PDT INFO remoteWorkerThread_3: SYNC 5000000061 done in 0.005 seconds 2017-06-14 23:10:28 PDT INFO remoteWorkerThread_2: SYNC 5000000079 done in 0.005 seconds 2017-06-14 23:10:33 PDT INFO remoteWorkerThread_3: SYNC 5000000062 done in 0.005 seconds 2017-06-14 23:10:37 PDT INFO remoteWorkerThread_2: SYNC 5000000080 done in 0.005 seconds 2017-06-14 23:10:41 PDT INFO remoteWorkerThread_3: SYNC 5000000063 done in 0.005 seconds 2017-06-14 23:10:46 PDT INFO remoteWorkerThread_2: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:10:52 PDT INFO remoteWorkerThread_3: SYNC 5000000064 done in 0.004 seconds 2017-06-14 23:10:57 PDT INFO remoteWorkerThread_2: SYNC 5000000082 done in 0.004 seconds 2017-06-14 23:11:03 PDT INFO remoteWorkerThread_3: SYNC 5000000065 done in 0.004 seconds 2017-06-14 23:11:08 PDT INFO remoteWorkerThread_2: SYNC 5000000083 done in 0.005 seconds 2017-06-14 23:11:14 PDT INFO remoteWorkerThread_3: SYNC 5000000066 done in 0.004 seconds 2017-06-14 23:11:19 PDT INFO remoteWorkerThread_2: SYNC 5000000084 done in 0.005 seconds 2017-06-14 23:11:23 PDT INFO remoteWorkerThread_2: SYNC 5000000085 done in 0.005 seconds 2017-06-14 23:11:25 PDT INFO remoteWorkerThread_3: SYNC 5000000067 done in 0.005 seconds 2017-06-14 23:11:34 PDT INFO remoteWorkerThread_2: SYNC 5000000086 done in 0.006 seconds 2017-06-14 23:11:38 PDT INFO remoteWorkerThread_3: SYNC 5000000068 done in 0.005 seconds 2017-06-14 23:11:42 PDT INFO remoteWorkerThread_3: SYNC 5000000069 done in 0.005 seconds 2017-06-14 23:11:46 PDT INFO remoteWorkerThread_2: SYNC 5000000087 done in 0.005 seconds 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: connection for provider 3 terminated 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 3 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 23:11:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:11:54 PDT INFO remoteWorkerThread_3: SYNC 5000000070 done in 0.005 seconds 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_3: connection for provider 3 terminated 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 3 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_3: connection for provider 2 terminated 2017-06-14 23:11:54 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 2 2017-06-14 23:11:58 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:11:58 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:11:58 PDT INFO remoteWorkerThread_2: SYNC 5000000088 done in 0.012 seconds 2017-06-14 23:12:02 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:12:02 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:02 PDT INFO remoteWorkerThread_3: SYNC 5000000071 done in 0.020 seconds 2017-06-14 23:12:06 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:12:06 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:07 PDT INFO remoteWorkerThread_2: SYNC 5000000089 done in 0.278 seconds 2017-06-14 23:12:13 PDT INFO remoteWorkerThread_3: SYNC 5000000072 done in 0.003 seconds 2017-06-14 23:12:24 PDT INFO remoteWorkerThread_3: SYNC 5000000073 done in 0.003 seconds 2017-06-14 23:12:33 PDT INFO remoteWorkerThread_2: SYNC 5000000090 done in 0.005 seconds 2017-06-14 23:12:37 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:12:37 PDT INFO remoteWorkerThread_2: SYNC 5000000091 done in 0.004 seconds 2017-06-14 23:12:37 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:38 PDT INFO remoteWorkerThread_3: SYNC 5000000074 done in 0.522 seconds 2017-06-14 23:12:41 PDT INFO remoteWorkerThread_3: SYNC 5000000075 done in 0.005 seconds 2017-06-14 23:12:52 PDT INFO remoteWorkerThread_3: SYNC 5000000076 done in 0.005 seconds 2017-06-14 23:12:55 PDT INFO remoteWorkerThread_2: SYNC 5000000092 done in 0.005 seconds 2017-06-14 23:13:03 PDT INFO remoteWorkerThread_2: SYNC 5000000093 done in 0.016 seconds 2017-06-14 23:13:09 PDT INFO remoteWorkerThread_3: SYNC 5000000077 done in 0.005 seconds 2017-06-14 23:13:10 PDT INFO remoteWorkerThread_3: SYNC 5000000078 done in 0.005 seconds 2017-06-14 23:13:14 PDT INFO remoteWorkerThread_2: SYNC 5000000094 done in 0.005 seconds 2017-06-14 23:13:21 PDT INFO remoteWorkerThread_3: SYNC 5000000079 done in 0.005 seconds 2017-06-14 23:13:25 PDT INFO remoteWorkerThread_2: SYNC 5000000095 done in 0.004 seconds 2017-06-14 23:13:32 PDT INFO remoteWorkerThread_3: SYNC 5000000080 done in 0.005 seconds 2017-06-14 23:13:36 PDT INFO remoteWorkerThread_2: SYNC 5000000096 done in 0.012 seconds 2017-06-14 23:13:43 PDT INFO remoteWorkerThread_3: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:13:47 PDT INFO remoteWorkerThread_2: SYNC 5000000097 done in 0.089 seconds 2017-06-14 23:13:54 PDT INFO remoteWorkerThread_3: SYNC 5000000082 done in 0.005 seconds 2017-06-14 23:13:58 PDT INFO remoteWorkerThread_2: SYNC 5000000098 done in 0.006 seconds 2017-06-14 23:14:06 PDT INFO remoteWorkerThread_3: SYNC 5000000083 done in 0.005 seconds 2017-06-14 23:14:10 PDT INFO remoteWorkerThread_3: SYNC 5000000084 done in 0.004 seconds 2017-06-14 23:14:11 PDT INFO remoteWorkerThread_2: SYNC 5000000099 done in 0.005 seconds 2017-06-14 23:14:21 PDT INFO remoteWorkerThread_3: SYNC 5000000085 done in 0.015 seconds 2017-06-14 23:14:21 PDT INFO remoteWorkerThread_2: SYNC 5000000100 done in 0.016 seconds 2017-06-14 23:14:32 PDT INFO remoteWorkerThread_3: SYNC 5000000086 done in 0.009 seconds 2017-06-14 23:14:35 PDT INFO remoteWorkerThread_2: SYNC 5000000101 done in 0.005 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2529 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2552 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2567 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=2568 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: log switch to sl_log_2 complete - truncate sl_log_1 CONTEXT: PL/pgSQL function _region_il.cleanupevent(interval) line 95 at assignment 2017-06-14 23:14:43 PDT INFO cleanupThread: 0.086 seconds for cleanupEvent() 2017-06-14 23:14:49 PDT INFO remoteWorkerThread_3: SYNC 5000000087 done in 0.005 seconds 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_3: SYNC 5000000088 done in 0.005 seconds 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: connection for provider 3 terminated 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 3 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: connection for provider 3 terminated 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 3 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: connection for provider 2 terminated 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 2 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=2 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=3 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=2 li_receiver=1 li_provider=3 2017-06-14 23:14:54 PDT CONFIG storeListen: li_origin=3 li_receiver=1 li_provider=2 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:14:54 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:54 PDT INFO remoteWorkerThread_2: SYNC 5000000102 done in 0.037 seconds 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:14:54 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:15:02 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:15:02 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:02 PDT INFO remoteWorkerThread_2: SYNC 5000000103 done in 0.013 seconds 2017-06-14 23:15:06 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:15:06 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:06 PDT INFO remoteWorkerThread_3: SYNC 5000000089 done in 0.013 seconds 2017-06-14 23:15:14 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:15:14 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:17 PDT INFO remoteWorkerThread_3: SYNC 5000000090 done in 2.438 seconds 2017-06-14 23:15:20 PDT INFO remoteWorkerThread_2: SYNC 5000000104 done in 0.003 seconds 2017-06-14 23:15:21 PDT INFO remoteWorkerThread_2: SYNC 5000000105 done in 0.002 seconds 2017-06-14 23:15:25 PDT INFO remoteWorkerThread_3: SYNC 5000000091 done in 0.016 seconds 2017-06-14 23:15:32 PDT INFO remoteWorkerThread_2: SYNC 5000000106 done in 0.002 seconds 2017-06-14 23:15:36 PDT INFO remoteWorkerThread_3: SYNC 5000000092 done in 0.005 seconds 2017-06-14 23:15:43 PDT INFO remoteWorkerThread_2: SYNC 5000000107 done in 0.003 seconds 2017-06-14 23:15:47 PDT INFO remoteWorkerThread_3: SYNC 5000000093 done in 0.006 seconds 2017-06-14 23:15:55 PDT INFO remoteWorkerThread_2: SYNC 5000000108 done in 0.003 seconds 2017-06-14 23:15:55 PDT INFO remoteWorkerThread_3: SYNC 5000000094 done in 0.013 seconds 2017-06-14 23:16:06 PDT INFO remoteWorkerThread_2: SYNC 5000000109 done in 0.003 seconds 2017-06-14 23:16:10 PDT INFO remoteWorkerThread_3: SYNC 5000000095 done in 0.004 seconds 2017-06-14 23:16:11 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:16:11 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:16:11 PDT INFO remoteWorkerThread_2: SYNC 5000000110 done in 0.058 seconds 2017-06-14 23:16:12 PDT INFO remoteWorkerThread_3: SYNC 5000000096 done in 0.012 seconds 2017-06-14 23:16:27 PDT INFO remoteWorkerThread_2: SYNC 5000000111 done in 0.008 seconds 2017-06-14 23:16:31 PDT INFO remoteWorkerThread_2: SYNC 5000000112 done in 0.013 seconds 2017-06-14 23:16:31 PDT INFO remoteWorkerThread_3: SYNC 5000000097 done in 0.037 seconds 2017-06-14 23:16:42 PDT INFO remoteWorkerThread_3: SYNC 5000000098 done in 0.006 seconds 2017-06-14 23:16:42 PDT INFO remoteWorkerThread_2: SYNC 5000000113 done in 0.006 seconds 2017-06-14 23:16:53 PDT INFO remoteWorkerThread_3: SYNC 5000000099 done in 0.005 seconds 2017-06-14 23:16:53 PDT INFO remoteWorkerThread_2: SYNC 5000000114 done in 0.005 seconds 2017-06-14 23:17:04 PDT INFO remoteWorkerThread_3: SYNC 5000000100 done in 0.005 seconds 2017-06-14 23:17:04 PDT INFO remoteWorkerThread_2: SYNC 5000000115 done in 0.006 seconds 2017-06-14 23:17:08 PDT INFO remoteWorkerThread_3: SYNC 5000000101 done in 0.005 seconds 2017-06-14 23:17:14 PDT INFO remoteWorkerThread_2: SYNC 5000000116 done in 0.004 seconds 2017-06-14 23:17:19 PDT INFO remoteWorkerThread_3: SYNC 5000000102 done in 0.005 seconds 2017-06-14 23:17:25 PDT INFO remoteWorkerThread_2: SYNC 5000000117 done in 0.005 seconds 2017-06-14 23:17:30 PDT INFO remoteWorkerThread_3: SYNC 5000000103 done in 0.005 seconds 2017-06-14 23:17:34 PDT INFO remoteWorkerThread_2: SYNC 5000000118 done in 0.013 seconds 2017-06-14 23:17:38 PDT INFO remoteWorkerThread_3: SYNC 5000000104 done in 0.012 seconds 2017-06-14 23:17:43 PDT INFO remoteWorkerThread_2: SYNC 5000000119 done in 0.005 seconds 2017-06-14 23:17:49 PDT INFO remoteWorkerThread_3: SYNC 5000000105 done in 0.004 seconds 2017-06-14 23:17:53 PDT INFO remoteWorkerThread_2: SYNC 5000000120 done in 0.005 seconds 2017-06-14 23:18:01 PDT INFO remoteWorkerThread_2: SYNC 5000000121 done in 0.005 seconds 2017-06-14 23:18:01 PDT INFO remoteWorkerThread_3: SYNC 5000000106 done in 0.006 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_2: SYNC 5000000122 done in 0.005 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_3: SYNC 5000000107 done in 0.006 seconds 2017-06-14 23:18:27 PDT INFO remoteWorkerThread_2: SYNC 5000000123 done in 0.005 seconds 2017-06-14 23:18:27 PDT INFO remoteWorkerThread_3: SYNC 5000000108 done in 0.006 seconds 2017-06-14 23:18:30 PDT INFO remoteWorkerThread_3: SYNC 5000000109 done in 0.005 seconds 2017-06-14 23:18:33 PDT INFO remoteWorkerThread_2: SYNC 5000000124 done in 0.004 seconds 2017-06-14 23:18:38 PDT INFO remoteWorkerThread_3: SYNC 5000000110 done in 0.004 seconds 2017-06-14 23:18:44 PDT INFO remoteWorkerThread_2: SYNC 5000000125 done in 0.004 seconds 2017-06-14 23:18:49 PDT INFO remoteWorkerThread_3: SYNC 5000000111 done in 0.005 seconds 2017-06-14 23:18:55 PDT INFO remoteWorkerThread_2: SYNC 5000000126 done in 0.005 seconds 2017-06-14 23:19:00 PDT INFO remoteWorkerThread_3: SYNC 5000000112 done in 0.011 seconds 2017-06-14 23:19:04 PDT INFO remoteWorkerThread_2: SYNC 5000000127 done in 0.005 seconds 2017-06-14 23:19:08 PDT INFO remoteWorkerThread_3: SYNC 5000000113 done in 0.005 seconds 2017-06-14 23:19:12 PDT INFO remoteWorkerThread_2: SYNC 5000000128 done in 0.005 seconds 2017-06-14 23:19:20 PDT INFO remoteWorkerThread_3: SYNC 5000000114 done in 0.004 seconds 2017-06-14 23:19:21 PDT INFO remoteWorkerThread_2: SYNC 5000000129 done in 0.004 seconds 2017-06-14 23:19:34 PDT INFO remoteWorkerThread_3: SYNC 5000000115 done in 0.005 seconds 2017-06-14 23:19:34 PDT INFO remoteWorkerThread_2: SYNC 5000000130 done in 0.006 seconds 2017-06-14 23:19:47 PDT INFO remoteWorkerThread_2: SYNC 5000000131 done in 0.005 seconds 2017-06-14 23:19:47 PDT INFO remoteWorkerThread_3: SYNC 5000000116 done in 0.005 seconds 2017-06-14 23:19:48 PDT INFO remoteWorkerThread_3: SYNC 5000000117 done in 0.005 seconds 2017-06-14 23:19:51 PDT INFO remoteWorkerThread_2: SYNC 5000000132 done in 0.005 seconds 2017-06-14 23:20:02 PDT INFO remoteWorkerThread_3: SYNC 5000000118 done in 0.005 seconds 2017-06-14 23:20:02 PDT INFO remoteWorkerThread_2: SYNC 5000000133 done in 0.006 seconds 2017-06-14 23:20:16 PDT INFO remoteWorkerThread_3: SYNC 5000000119 done in 0.005 seconds 2017-06-14 23:20:16 PDT INFO remoteWorkerThread_2: SYNC 5000000134 done in 0.006 seconds 2017-06-14 23:20:19 PDT INFO remoteWorkerThread_3: SYNC 5000000120 done in 0.005 seconds 2017-06-14 23:20:22 PDT INFO remoteWorkerThread_2: SYNC 5000000135 done in 0.005 seconds 2017-06-14 23:20:35 PDT INFO remoteWorkerThread_3: SYNC 5000000121 done in 0.005 seconds 2017-06-14 23:20:35 PDT INFO remoteWorkerThread_2: SYNC 5000000136 done in 0.006 seconds 2017-06-14 23:20:38 PDT INFO remoteWorkerThread_3: SYNC 5000000122 done in 0.005 seconds 2017-06-14 23:20:41 PDT INFO remoteWorkerThread_2: SYNC 5000000137 done in 0.005 seconds辅节点1查看Slony-I日志信息
$ cat /usr/local/pgsql/data/pg_log/node2/region_il-2017-06-14.log 2017-06-14 22:57:41 PDT CONFIG main: slon version 2.2.4 starting up 2017-06-14 22:57:41 PDT INFO slon: watchdog process started 2017-06-14 22:57:41 PDT CONFIG slon: watchdog ready - pid = 15535 2017-06-14 22:57:41 PDT CONFIG slon: worker process created - pid = 15536 2017-06-14 22:57:41 PDT CONFIG main: Integer option vac_frequency = 3 2017-06-14 22:57:41 PDT CONFIG main: Integer option log_level = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option sync_interval = 1000 2017-06-14 22:57:41 PDT CONFIG main: Integer option sync_interval_timeout = 10000 2017-06-14 22:57:41 PDT CONFIG main: Integer option sync_group_maxsize = 20 2017-06-14 22:57:41 PDT CONFIG main: Integer option syslog = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option quit_sync_provider = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option remote_listen_timeout = 300 2017-06-14 22:57:41 PDT CONFIG main: Integer option monitor_interval = 500 2017-06-14 22:57:41 PDT CONFIG main: Integer option explain_interval = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option tcp_keepalive_idle = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option tcp_keepalive_interval = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option tcp_keepalive_count = 0 2017-06-14 22:57:41 PDT CONFIG main: Integer option apply_cache_size = 100 2017-06-14 22:57:41 PDT CONFIG main: Boolean option log_pid = 0 2017-06-14 22:57:41 PDT CONFIG main: Boolean option log_timestamp = 1 2017-06-14 22:57:41 PDT CONFIG main: Boolean option tcp_keepalive = 1 2017-06-14 22:57:41 PDT CONFIG main: Boolean option monitor_threads = 1 2017-06-14 22:57:41 PDT CONFIG main: Real option real_placeholder = 0.000000 2017-06-14 22:57:41 PDT CONFIG main: String option cluster_name = region_il 2017-06-14 22:57:41 PDT CONFIG main: String option conn_info = host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888 2017-06-14 22:57:41 PDT CONFIG main: String option pid_file = /usr/local/pgsql/data/region_il_node2.pid 2017-06-14 22:57:41 PDT CONFIG main: String option log_timestamp_format = %Y-%m-%d %H:%M:%S %Z 2017-06-14 22:57:41 PDT CONFIG main: String option archive_dir = [NULL] 2017-06-14 22:57:41 PDT CONFIG main: String option sql_on_connection = [NULL] 2017-06-14 22:57:41 PDT CONFIG main: String option lag_interval = [NULL] 2017-06-14 22:57:41 PDT CONFIG main: String option command_on_logarchive = [NULL] 2017-06-14 22:57:41 PDT CONFIG main: String option syslog_facility = LOCAL0 2017-06-14 22:57:41 PDT CONFIG main: String option syslog_ident = slon 2017-06-14 22:57:41 PDT CONFIG main: String option cleanup_interval = 10 minutes 2017-06-14 22:57:41 PDT CONFIG main: local node id = 2 2017-06-14 22:57:41 PDT INFO main: main process started 2017-06-14 22:57:41 PDT CONFIG main: launching sched_start_mainloop 2017-06-14 22:57:41 PDT CONFIG main: loading current cluster configuration 2017-06-14 22:57:41 PDT CONFIG storeNode: no_id=1 no_comment='Node 1 - region_il@10.189.102.118' 2017-06-14 22:57:41 PDT CONFIG storeNode: no_id=3 no_comment='<event pending>' 2017-06-14 22:57:41 PDT CONFIG storePath: pa_server=1 pa_client=2 pa_conninfo="host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 22:57:41 PDT CONFIG storePath: pa_server=3 pa_client=2 pa_conninfo="host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG main: last local event sequence = 5000000002 2017-06-14 22:57:41 PDT CONFIG main: configuration complete - starting threads 2017-06-14 22:57:41 PDT INFO localListenThread: thread starts 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG enableNode: no_id=1 2017-06-14 22:57:41 PDT INFO main: running scheduler mainloop 2017-06-14 22:57:41 PDT CONFIG cleanupThread: thread starts 2017-06-14 22:57:41 PDT INFO syncThread: thread starts 2017-06-14 22:57:41 PDT INFO monitorThread: thread starts 2017-06-14 22:57:41 PDT INFO remoteListenThread_1: thread starts 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: thread starts 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG cleanupThread: bias = 60 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG storeNode: no_id=3 no_comment='Node 3 - region_il@10.189.100.226' - update node 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG enableNode: no_id=3 2017-06-14 22:57:41 PDT INFO remoteListenThread_3: thread starts 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_3: thread starts 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: SYNC 5000000008 done in 0.015 seconds 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: SYNC 5000000010 done in 0.002 seconds 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 22:57:41 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=3 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: SYNC 5000000014 done in 0.011 seconds 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 22:57:41 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 22:57:41 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: SYNC 5000000022 done in 0.012 seconds 2017-06-14 22:57:41 PDT INFO remoteWorkerThread_1: SYNC 5000000025 done in 0.002 seconds 2017-06-14 22:57:45 PDT INFO remoteWorkerThread_1: SYNC 5000000026 done in 0.003 seconds 2017-06-14 22:57:56 PDT INFO remoteWorkerThread_1: SYNC 5000000027 done in 0.003 seconds 2017-06-14 22:58:10 PDT INFO remoteWorkerThread_1: SYNC 5000000028 done in 0.003 seconds 2017-06-14 22:58:14 PDT INFO remoteWorkerThread_1: SYNC 5000000029 done in 0.003 seconds 2017-06-14 22:58:28 PDT INFO remoteWorkerThread_1: SYNC 5000000030 done in 0.003 seconds 2017-06-14 22:58:39 PDT INFO remoteWorkerThread_1: SYNC 5000000031 done in 0.003 seconds 2017-06-14 22:58:43 PDT INFO remoteWorkerThread_1: SYNC 5000000032 done in 0.003 seconds 2017-06-14 22:59:01 PDT INFO remoteWorkerThread_1: SYNC 5000000033 done in 0.003 seconds 2017-06-14 22:59:05 PDT INFO remoteWorkerThread_1: SYNC 5000000034 done in 0.003 seconds 2017-06-14 22:59:19 PDT INFO remoteWorkerThread_1: SYNC 5000000035 done in 0.003 seconds 2017-06-14 22:59:23 PDT INFO remoteWorkerThread_1: SYNC 5000000036 done in 0.002 seconds 2017-06-14 22:59:41 PDT INFO remoteWorkerThread_1: SYNC 5000000037 done in 0.003 seconds 2017-06-14 22:59:45 PDT INFO remoteWorkerThread_1: SYNC 5000000038 done in 0.002 seconds 2017-06-14 22:59:59 PDT INFO remoteWorkerThread_1: SYNC 5000000039 done in 0.003 seconds 2017-06-14 23:00:03 PDT INFO remoteWorkerThread_1: SYNC 5000000040 done in 0.003 seconds 2017-06-14 23:00:21 PDT INFO remoteWorkerThread_1: SYNC 5000000041 done in 0.003 seconds 2017-06-14 23:00:25 PDT INFO remoteWorkerThread_1: SYNC 5000000042 done in 0.002 seconds 2017-06-14 23:00:39 PDT INFO remoteWorkerThread_1: SYNC 5000000043 done in 0.003 seconds 2017-06-14 23:00:43 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:00:43 PDT INFO remoteWorkerThread_1: SYNC 5000000044 done in 0.003 seconds 2017-06-14 23:00:43 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:43 PDT INFO remoteWorkerThread_3: SYNC 5000000003 done in 0.014 seconds 2017-06-14 23:00:56 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:00:56 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:56 PDT INFO remoteWorkerThread_3: SYNC 5000000004 done in 0.022 seconds 2017-06-14 23:01:00 PDT INFO remoteWorkerThread_3: SYNC 5000000005 done in 0.013 seconds 2017-06-14 23:01:01 PDT INFO remoteWorkerThread_1: SYNC 5000000045 done in 0.003 seconds 2017-06-14 23:01:04 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:01:04 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:01:04 PDT INFO remoteWorkerThread_1: SYNC 5000000046 done in 0.014 seconds 2017-06-14 23:01:15 PDT INFO remoteWorkerThread_3: SYNC 5000000006 done in 0.005 seconds 2017-06-14 23:01:15 PDT INFO remoteWorkerThread_1: SYNC 5000000047 done in 0.006 seconds 2017-06-14 23:01:19 PDT INFO remoteWorkerThread_3: SYNC 5000000007 done in 0.004 seconds 2017-06-14 23:01:26 PDT INFO remoteWorkerThread_1: SYNC 5000000048 done in 0.005 seconds 2017-06-14 23:01:30 PDT INFO remoteWorkerThread_3: SYNC 5000000008 done in 0.005 seconds 2017-06-14 23:01:37 PDT INFO remoteWorkerThread_1: SYNC 5000000049 done in 0.005 seconds 2017-06-14 23:01:41 PDT INFO remoteWorkerThread_3: SYNC 5000000009 done in 0.005 seconds 2017-06-14 23:01:49 PDT INFO remoteWorkerThread_1: SYNC 5000000050 done in 0.013 seconds 2017-06-14 23:01:53 PDT INFO remoteWorkerThread_3: SYNC 5000000010 done in 0.005 seconds 2017-06-14 23:01:53 PDT INFO remoteWorkerThread_1: SYNC 5000000051 done in 0.006 seconds 2017-06-14 23:02:01 PDT INFO remoteWorkerThread_3: SYNC 5000000011 done in 0.006 seconds 2017-06-14 23:02:07 PDT INFO remoteWorkerThread_1: SYNC 5000000052 done in 0.005 seconds 2017-06-14 23:02:11 PDT INFO remoteWorkerThread_3: SYNC 5000000012 done in 0.005 seconds 2017-06-14 23:02:15 PDT INFO remoteWorkerThread_1: SYNC 5000000053 done in 0.005 seconds 2017-06-14 23:02:19 PDT INFO remoteWorkerThread_3: SYNC 5000000013 done in 0.005 seconds 2017-06-14 23:02:26 PDT INFO remoteWorkerThread_1: SYNC 5000000054 done in 0.005 seconds 2017-06-14 23:02:30 PDT INFO remoteWorkerThread_3: SYNC 5000000014 done in 0.005 seconds 2017-06-14 23:02:37 PDT INFO remoteWorkerThread_1: SYNC 5000000055 done in 0.005 seconds 2017-06-14 23:02:41 PDT INFO remoteWorkerThread_3: SYNC 5000000015 done in 0.005 seconds 2017-06-14 23:02:45 PDT INFO remoteWorkerThread_1: SYNC 5000000056 done in 0.006 seconds 2017-06-14 23:02:49 PDT INFO remoteWorkerThread_3: SYNC 5000000016 done in 0.006 seconds 2017-06-14 23:02:55 PDT INFO remoteWorkerThread_1: SYNC 5000000057 done in 0.005 seconds 2017-06-14 23:03:00 PDT INFO remoteWorkerThread_3: SYNC 5000000017 done in 0.005 seconds 2017-06-14 23:03:06 PDT INFO remoteWorkerThread_1: SYNC 5000000058 done in 0.009 seconds 2017-06-14 23:03:11 PDT INFO remoteWorkerThread_3: SYNC 5000000018 done in 0.004 seconds 2017-06-14 23:03:17 PDT CONFIG storeSet: set_id=1 set_origin=1 set_comment='Set 1 (il_longlive_schema) for region_il' 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: connection for provider 3 terminated 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 3 2017-06-14 23:03:17 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:17 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000060 done in 0.014 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000062 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000064 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000066 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000068 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000070 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000072 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000074 done in 0.002 seconds 2017-06-14 23:03:17 PDT INFO remoteWorkerThread_1: SYNC 5000000076 done in 0.002 seconds 2017-06-14 23:03:21 PDT INFO remoteWorkerThread_3: SYNC 5000000019 done in 0.005 seconds 2017-06-14 23:03:25 PDT INFO remoteWorkerThread_1: SYNC 5000000077 done in 0.002 seconds 2017-06-14 23:03:29 PDT INFO remoteWorkerThread_3: SYNC 5000000020 done in 0.005 seconds 2017-06-14 23:03:36 PDT INFO remoteWorkerThread_1: SYNC 5000000078 done in 0.003 seconds 2017-06-14 23:03:40 PDT INFO remoteWorkerThread_3: SYNC 5000000021 done in 0.005 seconds 2017-06-14 23:03:44 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:44 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:03:44 PDT INFO remoteWorkerThread_1: SYNC 5000000079 done in 0.015 seconds 2017-06-14 23:03:54 PDT INFO remoteWorkerThread_1: SYNC 5000000080 done in 0.005 seconds 2017-06-14 23:03:58 PDT INFO remoteWorkerThread_3: SYNC 5000000022 done in 0.005 seconds 2017-06-14 23:04:01 PDT INFO remoteWorkerThread_3: SYNC 5000000023 done in 0.005 seconds 2017-06-14 23:04:06 PDT INFO remoteWorkerThread_1: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:04:12 PDT INFO remoteWorkerThread_3: SYNC 5000000024 done in 0.005 seconds 2017-06-14 23:04:16 PDT INFO remoteWorkerThread_1: SYNC 5000000082 done in 0.005 seconds 2017-06-14 23:04:20 PDT INFO remoteWorkerThread_3: SYNC 5000000025 done in 0.005 seconds 2017-06-14 23:04:24 PDT INFO remoteWorkerThread_1: SYNC 5000000083 done in 0.005 seconds 2017-06-14 23:04:32 PDT INFO remoteWorkerThread_3: SYNC 5000000026 done in 0.005 seconds 2017-06-14 23:04:33 PDT INFO remoteWorkerThread_1: SYNC 5000000084 done in 0.004 seconds 2017-06-14 23:04:44 PDT INFO remoteWorkerThread_3: SYNC 5000000027 done in 0.005 seconds 2017-06-14 23:04:44 PDT INFO remoteWorkerThread_1: SYNC 5000000085 done in 0.006 seconds 2017-06-14 23:04:54 PDT INFO remoteWorkerThread_3: SYNC 5000000028 done in 0.005 seconds 2017-06-14 23:04:58 PDT INFO remoteWorkerThread_1: SYNC 5000000086 done in 0.005 seconds 2017-06-14 23:05:01 PDT INFO remoteWorkerThread_3: SYNC 5000000029 done in 0.006 seconds 2017-06-14 23:05:05 PDT INFO remoteWorkerThread_1: SYNC 5000000087 done in 0.005 seconds 2017-06-14 23:05:13 PDT INFO remoteWorkerThread_3: SYNC 5000000030 done in 0.005 seconds 2017-06-14 23:05:14 PDT INFO remoteWorkerThread_1: SYNC 5000000088 done in 0.005 seconds 2017-06-14 23:05:25 PDT INFO remoteWorkerThread_3: SYNC 5000000031 done in 0.005 seconds 2017-06-14 23:05:25 PDT INFO remoteWorkerThread_1: SYNC 5000000089 done in 0.006 seconds 2017-06-14 23:05:29 PDT INFO remoteWorkerThread_3: SYNC 5000000032 done in 0.005 seconds 2017-06-14 23:05:36 PDT INFO remoteWorkerThread_1: SYNC 5000000090 done in 0.006 seconds 2017-06-14 23:05:40 PDT INFO remoteWorkerThread_3: SYNC 5000000033 done in 0.004 seconds 2017-06-14 23:05:47 PDT INFO remoteWorkerThread_1: SYNC 5000000091 done in 0.005 seconds 2017-06-14 23:05:51 PDT INFO remoteWorkerThread_3: SYNC 5000000034 done in 0.004 seconds 2017-06-14 23:05:58 PDT INFO remoteWorkerThread_1: SYNC 5000000092 done in 0.005 seconds 2017-06-14 23:06:02 PDT INFO remoteWorkerThread_3: SYNC 5000000035 done in 0.005 seconds 2017-06-14 23:06:09 PDT INFO remoteWorkerThread_1: SYNC 5000000093 done in 0.005 seconds 2017-06-14 23:06:13 PDT INFO remoteWorkerThread_1: SYNC 5000000094 done in 0.005 seconds 2017-06-14 23:06:13 PDT INFO remoteWorkerThread_3: SYNC 5000000036 done in 0.004 seconds 2017-06-14 23:06:21 PDT INFO remoteWorkerThread_3: SYNC 5000000037 done in 0.004 seconds 2017-06-14 23:06:27 PDT INFO remoteWorkerThread_1: SYNC 5000000095 done in 0.004 seconds 2017-06-14 23:06:31 PDT INFO remoteWorkerThread_3: SYNC 5000000038 done in 0.005 seconds 2017-06-14 23:06:35 PDT INFO remoteWorkerThread_1: SYNC 5000000096 done in 0.005 seconds 2017-06-14 23:06:39 PDT INFO remoteWorkerThread_3: SYNC 5000000039 done in 0.004 seconds 2017-06-14 23:06:46 PDT INFO remoteWorkerThread_1: SYNC 5000000097 done in 0.005 seconds 2017-06-14 23:06:50 PDT INFO remoteWorkerThread_3: SYNC 5000000040 done in 0.004 seconds 2017-06-14 23:06:57 PDT INFO remoteWorkerThread_1: SYNC 5000000098 done in 0.005 seconds 2017-06-14 23:07:01 PDT INFO remoteWorkerThread_3: SYNC 5000000041 done in 0.005 seconds 2017-06-14 23:07:08 PDT INFO remoteWorkerThread_1: SYNC 5000000099 done in 0.004 seconds 2017-06-14 23:07:12 PDT INFO remoteWorkerThread_3: SYNC 5000000042 done in 0.005 seconds 2017-06-14 23:07:13 PDT INFO remoteWorkerThread_1: SYNC 5000000100 done in 0.004 seconds 2017-06-14 23:07:22 PDT INFO remoteWorkerThread_3: SYNC 5000000043 done in 0.005 seconds 2017-06-14 23:07:27 PDT INFO remoteWorkerThread_1: SYNC 5000000101 done in 0.030 seconds 2017-06-14 23:07:31 PDT INFO remoteWorkerThread_3: SYNC 5000000044 done in 0.005 seconds 2017-06-14 23:07:35 PDT INFO remoteWorkerThread_1: SYNC 5000000102 done in 0.004 seconds 2017-06-14 23:07:40 PDT INFO remoteWorkerThread_3: SYNC 5000000045 done in 0.009 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15654 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15655 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15658 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: Logswitch to sl_log_2 initiated CONTEXT: SQL statement "SELECT "_region_il".logswitch_start()" PL/pgSQL function _region_il.cleanupevent(interval) line 97 at PERFORM 2017-06-14 23:07:42 PDT INFO cleanupThread: 0.019 seconds for cleanupEvent() 2017-06-14 23:07:46 PDT INFO remoteWorkerThread_1: SYNC 5000000103 done in 0.009 seconds 2017-06-14 23:07:51 PDT INFO remoteWorkerThread_3: SYNC 5000000046 done in 0.005 seconds 2017-06-14 23:07:57 PDT INFO remoteWorkerThread_1: SYNC 5000000104 done in 0.010 seconds 2017-06-14 23:08:02 PDT INFO remoteWorkerThread_3: SYNC 5000000047 done in 0.009 seconds 2017-06-14 23:08:08 PDT INFO remoteWorkerThread_1: SYNC 5000000105 done in 0.012 seconds 2017-06-14 23:08:13 PDT INFO remoteWorkerThread_1: SYNC 5000000106 done in 0.008 seconds 2017-06-14 23:08:13 PDT INFO remoteWorkerThread_3: SYNC 5000000048 done in 0.013 seconds 2017-06-14 23:08:24 PDT INFO remoteWorkerThread_3: SYNC 5000000049 done in 0.009 seconds 2017-06-14 23:08:24 PDT INFO remoteWorkerThread_1: SYNC 5000000107 done in 0.010 seconds 2017-06-14 23:08:35 PDT INFO remoteWorkerThread_3: SYNC 5000000050 done in 0.005 seconds 2017-06-14 23:08:39 PDT INFO remoteWorkerThread_1: SYNC 5000000108 done in 0.006 seconds 2017-06-14 23:08:39 PDT INFO remoteWorkerThread_3: SYNC 5000000051 done in 0.005 seconds 2017-06-14 23:08:43 PDT INFO remoteWorkerThread_1: SYNC 5000000109 done in 0.005 seconds 2017-06-14 23:08:54 PDT INFO remoteWorkerThread_3: SYNC 5000000052 done in 0.006 seconds 2017-06-14 23:08:54 PDT INFO remoteWorkerThread_1: SYNC 5000000110 done in 0.006 seconds 2017-06-14 23:09:04 PDT INFO remoteWorkerThread_3: SYNC 5000000053 done in 0.005 seconds 2017-06-14 23:09:08 PDT INFO remoteWorkerThread_1: SYNC 5000000111 done in 0.005 seconds 2017-06-14 23:09:11 PDT INFO remoteWorkerThread_3: SYNC 5000000054 done in 0.005 seconds 2017-06-14 23:09:16 PDT INFO remoteWorkerThread_1: SYNC 5000000112 done in 0.005 seconds 2017-06-14 23:09:22 PDT INFO remoteWorkerThread_3: SYNC 5000000055 done in 0.005 seconds 2017-06-14 23:09:27 PDT INFO remoteWorkerThread_1: SYNC 5000000113 done in 0.005 seconds 2017-06-14 23:09:33 PDT INFO remoteWorkerThread_3: SYNC 5000000056 done in 0.005 seconds 2017-06-14 23:09:34 PDT INFO remoteWorkerThread_1: SYNC 5000000114 done in 0.005 seconds 2017-06-14 23:09:45 PDT INFO remoteWorkerThread_1: SYNC 5000000115 done in 0.005 seconds 2017-06-14 23:09:48 PDT INFO remoteWorkerThread_3: SYNC 5000000057 done in 0.005 seconds 2017-06-14 23:09:49 PDT INFO remoteWorkerThread_3: SYNC 5000000058 done in 0.005 seconds 2017-06-14 23:09:59 PDT INFO remoteWorkerThread_1: SYNC 5000000116 done in 0.005 seconds 2017-06-14 23:10:00 PDT INFO remoteWorkerThread_3: SYNC 5000000059 done in 0.005 seconds 2017-06-14 23:10:04 PDT INFO remoteWorkerThread_1: SYNC 5000000117 done in 0.005 seconds 2017-06-14 23:10:10 PDT INFO remoteWorkerThread_3: SYNC 5000000060 done in 0.005 seconds 2017-06-14 23:10:14 PDT INFO remoteWorkerThread_1: SYNC 5000000118 done in 0.005 seconds 2017-06-14 23:10:22 PDT INFO remoteWorkerThread_3: SYNC 5000000061 done in 0.005 seconds 2017-06-14 23:10:28 PDT INFO remoteWorkerThread_1: SYNC 5000000119 done in 0.005 seconds 2017-06-14 23:10:33 PDT INFO remoteWorkerThread_3: SYNC 5000000062 done in 0.005 seconds 2017-06-14 23:10:39 PDT INFO remoteWorkerThread_1: SYNC 5000000120 done in 0.004 seconds 2017-06-14 23:10:43 PDT INFO remoteWorkerThread_3: SYNC 5000000063 done in 0.005 seconds 2017-06-14 23:10:43 PDT INFO remoteWorkerThread_1: SYNC 5000000121 done in 0.005 seconds 2017-06-14 23:10:52 PDT INFO remoteWorkerThread_3: SYNC 5000000064 done in 0.005 seconds 2017-06-14 23:10:57 PDT INFO remoteWorkerThread_1: SYNC 5000000122 done in 0.004 seconds 2017-06-14 23:11:03 PDT INFO remoteWorkerThread_3: SYNC 5000000065 done in 0.005 seconds 2017-06-14 23:11:08 PDT INFO remoteWorkerThread_1: SYNC 5000000123 done in 0.005 seconds 2017-06-14 23:11:14 PDT INFO remoteWorkerThread_3: SYNC 5000000066 done in 0.005 seconds 2017-06-14 23:11:19 PDT INFO remoteWorkerThread_1: SYNC 5000000124 done in 0.005 seconds 2017-06-14 23:11:25 PDT INFO remoteWorkerThread_3: SYNC 5000000067 done in 0.005 seconds 2017-06-14 23:11:30 PDT INFO remoteWorkerThread_1: SYNC 5000000125 done in 0.005 seconds 2017-06-14 23:11:34 PDT INFO remoteWorkerThread_1: SYNC 5000000126 done in 0.005 seconds 2017-06-14 23:11:36 PDT INFO remoteWorkerThread_3: SYNC 5000000068 done in 0.005 seconds 2017-06-14 23:11:40 PDT INFO remoteWorkerThread_3: SYNC 5000000069 done in 0.004 seconds 2017-06-14 23:11:45 PDT INFO remoteWorkerThread_1: SYNC 5000000127 done in 0.005 seconds 2017-06-14 23:11:51 PDT INFO remoteWorkerThread_3: SYNC 5000000070 done in 0.004 seconds 2017-06-14 23:11:56 PDT CONFIG storeSubscribe: sub_set=1 sub_provider=1 sub_forward='t' 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=3 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: connection for provider 3 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 3 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_3: connection for provider 1 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 1 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_3: connection for provider 3 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 3 2017-06-14 23:11:56 PDT INFO copy_set 1 - omit=f - bool=0 2017-06-14 23:11:56 PDT INFO omit is FALSE 2017-06-14 23:11:56 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: connected to provider DB 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."entity" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."entity_result" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."result" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."result_logbook" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: all tables for set 1 found on subscriber 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."entity_entity_id_seq" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."entity_result_id_seq" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."result_id_seq" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."result_logbook_record_id_seq" 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."entity" 2017-06-14 23:11:57 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."entity" NOTICE: truncate of "il_longlive"."entity" succeeded 2017-06-14 23:12:02 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:12:02 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:02 PDT INFO remoteWorkerThread_3: SYNC 5000000071 done in 0.029 seconds 2017-06-14 23:12:13 PDT INFO remoteWorkerThread_3: SYNC 5000000072 done in 0.003 seconds 2017-06-14 23:12:28 PDT INFO remoteWorkerThread_3: SYNC 5000000073 done in 0.016 seconds 2017-06-14 23:12:33 PDT CONFIG remoteWorkerThread_1: 125045870 bytes copied for table "il_longlive"."entity" 2017-06-14 23:12:35 PDT INFO remoteWorkerThread_3: SYNC 5000000074 done in 0.010 seconds 2017-06-14 23:12:43 PDT CONFIG remoteWorkerThread_1: 46.468 seconds to copy table "il_longlive"."entity" 2017-06-14 23:12:43 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."entity_result" 2017-06-14 23:12:43 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."entity_result" NOTICE: truncate of "il_longlive"."entity_result" succeeded 2017-06-14 23:12:46 PDT INFO remoteWorkerThread_3: SYNC 5000000075 done in 0.003 seconds 2017-06-14 23:12:50 PDT INFO remoteWorkerThread_3: SYNC 5000000076 done in 0.003 seconds 2017-06-14 23:13:12 PDT CONFIG remoteWorkerThread_1: 66996342 bytes copied for table "il_longlive"."entity_result" 2017-06-14 23:13:14 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:13:14 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:13:14 PDT INFO remoteWorkerThread_3: SYNC 5000000078 done in 0.025 seconds 2017-06-14 23:13:23 PDT INFO remoteWorkerThread_3: SYNC 5000000079 done in 0.013 seconds 2017-06-14 23:13:32 PDT INFO remoteWorkerThread_3: SYNC 5000000080 done in 0.005 seconds 2017-06-14 23:13:41 PDT INFO remoteWorkerThread_3: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:13:55 PDT INFO remoteWorkerThread_3: SYNC 5000000082 done in 0.005 seconds 2017-06-14 23:13:57 PDT CONFIG remoteWorkerThread_1: 74.255 seconds to copy table "il_longlive"."entity_result" 2017-06-14 23:13:57 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."result" 2017-06-14 23:13:57 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."result" NOTICE: truncate of "il_longlive"."result" succeeded 2017-06-14 23:14:07 PDT INFO remoteWorkerThread_3: SYNC 5000000083 done in 0.005 seconds 2017-06-14 23:14:20 PDT INFO remoteWorkerThread_3: SYNC 5000000084 done in 0.020 seconds 2017-06-14 23:14:23 PDT INFO remoteWorkerThread_3: SYNC 5000000085 done in 0.005 seconds 2017-06-14 23:14:29 PDT CONFIG remoteWorkerThread_1: 105961503 bytes copied for table "il_longlive"."result" 2017-06-14 23:14:33 PDT INFO remoteWorkerThread_3: SYNC 5000000086 done in 0.005 seconds 2017-06-14 23:14:41 PDT INFO remoteWorkerThread_3: SYNC 5000000087 done in 0.005 seconds 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: 52.781 seconds to copy table "il_longlive"."result" 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."result_logbook" 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."result_logbook" NOTICE: truncate of "il_longlive"."result_logbook" succeeded 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: 79 bytes copied for table "il_longlive"."result_logbook" 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: 0.322 seconds to copy table "il_longlive"."result_logbook" 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 103 ("il_longlive"."result_id_seq") to 2611527 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 102 ("il_longlive"."entity_result_id_seq") to 2611527 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 104 ("il_longlive"."result_logbook_record_id_seq") to 1 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 101 ("il_longlive"."entity_entity_id_seq") to 2528612 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: 0.002 seconds to copy sequences 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: copy_set SYNC found, use event seqno 5000000130. 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: 0.005 seconds to build initial setsync status 2017-06-14 23:14:50 PDT INFO copy_set 1 done in 173.913 seconds 2017-06-14 23:14:50 PDT CONFIG enableSubscription: sub_set=1 2017-06-14 23:14:50 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 23:14:50 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 23:14:50 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 1 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_3: connection for provider 3 terminated 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 3 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_3: connection for provider 1 terminated 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 1 2017-06-14 23:14:50 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: SYNC 5000000131 done in 0.053 seconds 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:14:50 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 1 2017-06-14 23:14:50 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: SYNC 5000000135 done in 0.016 seconds 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: SYNC 5000000143 done in 0.005 seconds 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: SYNC 5000000147 done in 0.005 seconds 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 1 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: connection for provider 3 terminated 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 3 2017-06-14 23:14:52 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=3 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=3 li_receiver=2 li_provider=1 2017-06-14 23:14:52 PDT CONFIG storeListen: li_origin=1 li_receiver=2 li_provider=1 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 1 2017-06-14 23:14:52 PDT INFO remoteWorkerThread_3: SYNC 5000000088 done in 0.013 seconds 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_3: connection for provider 1 terminated 2017-06-14 23:14:52 PDT CONFIG remoteWorkerThread_3: disconnecting from data provider 1 2017-06-14 23:15:00 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:00 PDT INFO remoteWorkerThread_1: SYNC 5000000150 done in 0.006 seconds 2017-06-14 23:15:05 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:15:05 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:05 PDT INFO remoteWorkerThread_3: SYNC 5000000089 done in 0.054 seconds 2017-06-14 23:15:11 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:11 PDT INFO remoteWorkerThread_1: SYNC 5000000151 done in 0.004 seconds 2017-06-14 23:15:15 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:15 PDT INFO remoteWorkerThread_1: SYNC 5000000152 done in 0.004 seconds 2017-06-14 23:15:16 PDT INFO remoteWorkerThread_3: SYNC 5000000090 done in 0.005 seconds 2017-06-14 23:15:26 PDT CONFIG remoteWorkerThread_3: update provider configuration 2017-06-14 23:15:26 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:26 PDT INFO remoteWorkerThread_1: SYNC 5000000153 done in 0.007 seconds 2017-06-14 23:15:26 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:26 PDT INFO remoteWorkerThread_3: SYNC 5000000091 done in 0.025 seconds 2017-06-14 23:15:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:37 PDT INFO remoteWorkerThread_1: SYNC 5000000154 done in 0.005 seconds 2017-06-14 23:15:37 PDT INFO remoteWorkerThread_3: SYNC 5000000092 done in 0.006 seconds 2017-06-14 23:15:47 PDT INFO remoteWorkerThread_3: SYNC 5000000093 done in 0.005 seconds 2017-06-14 23:15:51 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:51 PDT INFO remoteWorkerThread_1: SYNC 5000000155 done in 0.004 seconds 2017-06-14 23:15:55 PDT INFO remoteWorkerThread_3: SYNC 5000000094 done in 0.005 seconds 2017-06-14 23:15:59 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:15:59 PDT INFO remoteWorkerThread_1: SYNC 5000000156 done in 0.005 seconds 2017-06-14 23:16:09 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:09 PDT INFO remoteWorkerThread_1: SYNC 5000000157 done in 0.004 seconds 2017-06-14 23:16:12 PDT INFO remoteWorkerThread_3: SYNC 5000000095 done in 0.009 seconds 2017-06-14 23:16:13 PDT INFO remoteWorkerThread_3: SYNC 5000000096 done in 0.015 seconds 2017-06-14 23:16:17 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:17 PDT INFO remoteWorkerThread_1: SYNC 5000000158 done in 0.004 seconds 2017-06-14 23:16:28 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:28 PDT INFO remoteWorkerThread_1: SYNC 5000000159 done in 0.004 seconds 2017-06-14 23:16:31 PDT INFO remoteWorkerThread_3: SYNC 5000000097 done in 0.005 seconds 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: SYNC 5000000160 done in 0.004 seconds 2017-06-14 23:16:39 PDT INFO remoteWorkerThread_3: SYNC 5000000098 done in 0.005 seconds 2017-06-14 23:16:46 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:46 PDT INFO remoteWorkerThread_1: SYNC 5000000161 done in 0.004 seconds 2017-06-14 23:16:50 PDT INFO remoteWorkerThread_3: SYNC 5000000099 done in 0.005 seconds 2017-06-14 23:16:57 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:16:57 PDT INFO remoteWorkerThread_1: SYNC 5000000162 done in 0.004 seconds 2017-06-14 23:17:01 PDT INFO remoteWorkerThread_3: SYNC 5000000100 done in 0.008 seconds 2017-06-14 23:17:09 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:17:09 PDT INFO remoteWorkerThread_3: SYNC 5000000101 done in 0.006 seconds 2017-06-14 23:17:09 PDT INFO remoteWorkerThread_1: SYNC 5000000163 done in 0.005 seconds 2017-06-14 23:17:22 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:17:22 PDT INFO remoteWorkerThread_1: SYNC 5000000164 done in 0.005 seconds 2017-06-14 23:17:22 PDT INFO remoteWorkerThread_3: SYNC 5000000102 done in 0.005 seconds 2017-06-14 23:17:25 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:17:25 PDT INFO remoteWorkerThread_1: SYNC 5000000165 done in 0.004 seconds 2017-06-14 23:17:30 PDT INFO remoteWorkerThread_3: SYNC 5000000103 done in 0.005 seconds 2017-06-14 23:17:36 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:17:36 PDT INFO remoteWorkerThread_1: SYNC 5000000166 done in 0.004 seconds 2017-06-14 23:17:40 PDT INFO remoteWorkerThread_3: SYNC 5000000104 done in 0.013 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15559 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15656 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15675 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15683 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15825 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15838 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15853 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15939 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15940 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=15941 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: log switch to sl_log_2 complete - truncate sl_log_1 CONTEXT: PL/pgSQL function _region_il.cleanupevent(interval) line 95 at assignment 2017-06-14 23:17:42 PDT INFO cleanupThread: 0.047 seconds for cleanupEvent() 2017-06-14 23:17:48 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:17:48 PDT INFO remoteWorkerThread_3: SYNC 5000000105 done in 0.005 seconds 2017-06-14 23:17:48 PDT INFO remoteWorkerThread_1: SYNC 5000000167 done in 0.005 seconds 2017-06-14 23:18:01 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:01 PDT INFO remoteWorkerThread_3: SYNC 5000000106 done in 0.005 seconds 2017-06-14 23:18:01 PDT INFO remoteWorkerThread_1: SYNC 5000000168 done in 0.005 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_3: SYNC 5000000107 done in 0.006 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_1: SYNC 5000000169 done in 0.006 seconds 2017-06-14 23:18:17 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:17 PDT INFO remoteWorkerThread_1: SYNC 5000000170 done in 0.005 seconds 2017-06-14 23:18:20 PDT INFO remoteWorkerThread_3: SYNC 5000000108 done in 0.005 seconds 2017-06-14 23:18:28 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:28 PDT INFO remoteWorkerThread_3: SYNC 5000000109 done in 0.005 seconds 2017-06-14 23:18:28 PDT INFO remoteWorkerThread_1: SYNC 5000000171 done in 0.005 seconds 2017-06-14 23:18:41 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:41 PDT INFO remoteWorkerThread_1: SYNC 5000000172 done in 0.005 seconds 2017-06-14 23:18:41 PDT INFO remoteWorkerThread_3: SYNC 5000000110 done in 0.005 seconds 2017-06-14 23:18:54 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:54 PDT INFO remoteWorkerThread_3: SYNC 5000000111 done in 0.005 seconds 2017-06-14 23:18:54 PDT INFO remoteWorkerThread_1: SYNC 5000000173 done in 0.005 seconds 2017-06-14 23:18:57 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:18:57 PDT INFO remoteWorkerThread_1: SYNC 5000000174 done in 0.013 seconds 2017-06-14 23:19:00 PDT INFO remoteWorkerThread_3: SYNC 5000000112 done in 0.009 seconds 2017-06-14 23:19:08 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:19:08 PDT INFO remoteWorkerThread_3: SYNC 5000000113 done in 0.005 seconds 2017-06-14 23:19:08 PDT INFO remoteWorkerThread_1: SYNC 5000000175 done in 0.005 seconds 2017-06-14 23:19:21 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:19:21 PDT INFO remoteWorkerThread_3: SYNC 5000000114 done in 0.006 seconds 2017-06-14 23:19:21 PDT INFO remoteWorkerThread_1: SYNC 5000000176 done in 0.005 seconds 2017-06-14 23:19:34 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:19:34 PDT INFO remoteWorkerThread_3: SYNC 5000000115 done in 0.006 seconds 2017-06-14 23:19:34 PDT INFO remoteWorkerThread_1: SYNC 5000000177 done in 0.006 seconds 2017-06-14 23:19:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:19:37 PDT INFO remoteWorkerThread_1: SYNC 5000000178 done in 0.005 seconds 2017-06-14 23:19:40 PDT INFO remoteWorkerThread_3: SYNC 5000000116 done in 0.005 seconds 2017-06-14 23:19:48 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:19:48 PDT INFO remoteWorkerThread_3: SYNC 5000000117 done in 0.005 seconds 2017-06-14 23:19:48 PDT INFO remoteWorkerThread_1: SYNC 5000000179 done in 0.005 seconds 2017-06-14 23:20:01 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:01 PDT INFO remoteWorkerThread_1: SYNC 5000000180 done in 0.006 seconds 2017-06-14 23:20:01 PDT INFO remoteWorkerThread_3: SYNC 5000000118 done in 0.006 seconds 2017-06-14 23:20:14 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:14 PDT INFO remoteWorkerThread_1: SYNC 5000000181 done in 0.005 seconds 2017-06-14 23:20:14 PDT INFO remoteWorkerThread_3: SYNC 5000000119 done in 0.005 seconds 2017-06-14 23:20:17 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:17 PDT INFO remoteWorkerThread_1: SYNC 5000000182 done in 0.005 seconds 2017-06-14 23:20:18 PDT INFO remoteWorkerThread_3: SYNC 5000000120 done in 0.005 seconds 2017-06-14 23:20:27 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:27 PDT INFO remoteWorkerThread_1: SYNC 5000000183 done in 0.005 seconds 2017-06-14 23:20:32 PDT INFO remoteWorkerThread_3: SYNC 5000000121 done in 0.006 seconds 2017-06-14 23:20:38 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:38 PDT INFO remoteWorkerThread_1: SYNC 5000000184 done in 0.004 seconds 2017-06-14 23:20:39 PDT INFO remoteWorkerThread_3: SYNC 5000000122 done in 0.005 seconds 2017-06-14 23:20:53 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:20:53 PDT INFO remoteWorkerThread_3: SYNC 5000000123 done in 0.006 seconds 2017-06-14 23:20:53 PDT INFO remoteWorkerThread_1: SYNC 5000000185 done in 0.006 seconds 2017-06-14 23:21:01 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:01 PDT INFO remoteWorkerThread_3: SYNC 5000000124 done in 0.006 seconds 2017-06-14 23:21:01 PDT INFO remoteWorkerThread_1: SYNC 5000000186 done in 0.006 seconds 2017-06-14 23:21:09 PDT INFO remoteWorkerThread_3: SYNC 5000000125 done in 0.005 seconds 2017-06-14 23:21:14 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:14 PDT INFO remoteWorkerThread_1: SYNC 5000000187 done in 0.004 seconds 2017-06-14 23:21:15 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:15 PDT INFO remoteWorkerThread_1: SYNC 5000000188 done in 0.004 seconds 2017-06-14 23:21:20 PDT INFO remoteWorkerThread_3: SYNC 5000000126 done in 0.005 seconds 2017-06-14 23:21:26 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:26 PDT INFO remoteWorkerThread_1: SYNC 5000000189 done in 0.004 seconds 2017-06-14 23:21:31 PDT INFO remoteWorkerThread_3: SYNC 5000000127 done in 0.005 seconds 2017-06-14 23:21:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:37 PDT INFO remoteWorkerThread_1: SYNC 5000000190 done in 0.005 seconds 2017-06-14 23:21:42 PDT INFO remoteWorkerThread_3: SYNC 5000000128 done in 0.005 seconds 2017-06-14 23:21:48 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:21:48 PDT INFO remoteWorkerThread_1: SYNC 5000000191 done in 0.008 seconds 2017-06-14 23:21:52 PDT INFO remoteWorkerThread_3: SYNC 5000000129 done in 0.006 seconds 2017-06-14 23:22:00 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:00 PDT INFO remoteWorkerThread_1: SYNC 5000000192 done in 0.005 seconds 2017-06-14 23:22:00 PDT INFO remoteWorkerThread_3: SYNC 5000000130 done in 0.006 seconds 2017-06-14 23:22:13 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:13 PDT INFO remoteWorkerThread_3: SYNC 5000000131 done in 0.005 seconds 2017-06-14 23:22:13 PDT INFO remoteWorkerThread_1: SYNC 5000000193 done in 0.005 seconds 2017-06-14 23:22:16 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:16 PDT INFO remoteWorkerThread_1: SYNC 5000000194 done in 0.004 seconds 2017-06-14 23:22:21 PDT INFO remoteWorkerThread_3: SYNC 5000000132 done in 0.005 seconds 2017-06-14 23:22:27 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:27 PDT INFO remoteWorkerThread_1: SYNC 5000000195 done in 0.005 seconds 2017-06-14 23:22:32 PDT INFO remoteWorkerThread_3: SYNC 5000000133 done in 0.005 seconds 2017-06-14 23:22:38 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:38 PDT INFO remoteWorkerThread_1: SYNC 5000000196 done in 0.006 seconds 2017-06-14 23:22:42 PDT INFO remoteWorkerThread_3: SYNC 5000000134 done in 0.005 seconds 2017-06-14 23:22:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:50 PDT INFO remoteWorkerThread_1: SYNC 5000000197 done in 0.005 seconds 2017-06-14 23:22:54 PDT INFO remoteWorkerThread_3: SYNC 5000000135 done in 0.005 seconds 2017-06-14 23:22:55 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:22:55 PDT INFO remoteWorkerThread_1: SYNC 5000000198 done in 0.004 seconds 2017-06-14 23:22:59 PDT INFO remoteWorkerThread_3: SYNC 5000000136 done in 0.005 seconds 2017-06-14 23:23:10 PDT INFO remoteWorkerThread_3: SYNC 5000000137 done in 0.005 seconds 2017-06-14 23:23:13 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:23:13 PDT INFO remoteWorkerThread_1: SYNC 5000000199 done in 0.005 seconds 2017-06-14 23:23:21 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:23:21 PDT INFO remoteWorkerThread_1: SYNC 5000000200 done in 0.004 seconds 2017-06-14 23:23:25 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:23:25 PDT INFO remoteWorkerThread_3: SYNC 5000000138 done in 0.005 seconds 2017-06-14 23:23:25 PDT INFO remoteWorkerThread_1: SYNC 5000000201 done in 0.005 seconds 2017-06-14 23:23:34 PDT INFO remoteWorkerThread_3: SYNC 5000000139 done in 0.005 seconds 2017-06-14 23:23:39 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:23:39 PDT INFO remoteWorkerThread_1: SYNC 5000000202 done in 0.005 seconds 2017-06-14 23:23:43 PDT INFO remoteWorkerThread_3: SYNC 5000000140 done in 0.005 seconds 2017-06-14 23:23:47 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 1 2017-06-14 23:23:47 PDT INFO remoteWorkerThread_1: SYNC 5000000203 done in 0.004 seconds辅节点2查看Slony-I日志信息
$ cat /usr/local/pgsql/data/pg_log/node3/region_il-2017-06-14.log 2017-06-14 23:00:38 PDT CONFIG main: slon version 2.2.4 starting up 2017-06-14 23:00:38 PDT INFO slon: watchdog process started 2017-06-14 23:00:38 PDT CONFIG slon: watchdog ready - pid = 19212 2017-06-14 23:00:38 PDT CONFIG slon: worker process created - pid = 19213 2017-06-14 23:00:38 PDT CONFIG main: Integer option vac_frequency = 3 2017-06-14 23:00:38 PDT CONFIG main: Integer option log_level = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option sync_interval = 1000 2017-06-14 23:00:38 PDT CONFIG main: Integer option sync_interval_timeout = 10000 2017-06-14 23:00:38 PDT CONFIG main: Integer option sync_group_maxsize = 20 2017-06-14 23:00:38 PDT CONFIG main: Integer option syslog = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option quit_sync_provider = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option remote_listen_timeout = 300 2017-06-14 23:00:38 PDT CONFIG main: Integer option monitor_interval = 500 2017-06-14 23:00:38 PDT CONFIG main: Integer option explain_interval = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option tcp_keepalive_idle = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option tcp_keepalive_interval = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option tcp_keepalive_count = 0 2017-06-14 23:00:38 PDT CONFIG main: Integer option apply_cache_size = 100 2017-06-14 23:00:38 PDT CONFIG main: Boolean option log_pid = 0 2017-06-14 23:00:38 PDT CONFIG main: Boolean option log_timestamp = 1 2017-06-14 23:00:38 PDT CONFIG main: Boolean option tcp_keepalive = 1 2017-06-14 23:00:38 PDT CONFIG main: Boolean option monitor_threads = 1 2017-06-14 23:00:38 PDT CONFIG main: Real option real_placeholder = 0.000000 2017-06-14 23:00:38 PDT CONFIG main: String option cluster_name = region_il 2017-06-14 23:00:38 PDT CONFIG main: String option conn_info = host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888 2017-06-14 23:00:38 PDT CONFIG main: String option pid_file = /usr/local/pgsql/data/region_il_node3.pid 2017-06-14 23:00:38 PDT CONFIG main: String option log_timestamp_format = %Y-%m-%d %H:%M:%S %Z 2017-06-14 23:00:38 PDT CONFIG main: String option archive_dir = [NULL] 2017-06-14 23:00:38 PDT CONFIG main: String option sql_on_connection = [NULL] 2017-06-14 23:00:38 PDT CONFIG main: String option lag_interval = [NULL] 2017-06-14 23:00:38 PDT CONFIG main: String option command_on_logarchive = [NULL] 2017-06-14 23:00:38 PDT CONFIG main: String option syslog_facility = LOCAL0 2017-06-14 23:00:38 PDT CONFIG main: String option syslog_ident = slon 2017-06-14 23:00:38 PDT CONFIG main: String option cleanup_interval = 10 minutes 2017-06-14 23:00:38 PDT CONFIG main: local node id = 3 2017-06-14 23:00:38 PDT INFO main: main process started 2017-06-14 23:00:38 PDT CONFIG main: launching sched_start_mainloop 2017-06-14 23:00:38 PDT CONFIG main: loading current cluster configuration 2017-06-14 23:00:38 PDT CONFIG storeNode: no_id=1 no_comment='Node 1 - region_il@10.189.102.118' 2017-06-14 23:00:38 PDT CONFIG storeNode: no_id=2 no_comment='Node 2 - region_il@10.189.100.195' 2017-06-14 23:00:38 PDT CONFIG storePath: pa_server=1 pa_client=3 pa_conninfo="host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 23:00:38 PDT CONFIG storePath: pa_server=2 pa_client=3 pa_conninfo="host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" pa_connretry=10 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG main: last local event sequence = 5000000002 2017-06-14 23:00:38 PDT CONFIG main: configuration complete - starting threads 2017-06-14 23:00:38 PDT INFO localListenThread: thread starts 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG enableNode: no_id=1 2017-06-14 23:00:38 PDT CONFIG enableNode: no_id=2 2017-06-14 23:00:38 PDT INFO main: running scheduler mainloop 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: thread starts 2017-06-14 23:00:38 PDT INFO remoteListenThread_2: thread starts 2017-06-14 23:00:38 PDT CONFIG cleanupThread: thread starts 2017-06-14 23:00:38 PDT INFO syncThread: thread starts 2017-06-14 23:00:38 PDT INFO monitorThread: thread starts 2017-06-14 23:00:38 PDT INFO remoteListenThread_1: thread starts 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: thread starts 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG cleanupThread: bias = 60 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: SYNC 5000000003 done in 0.019 seconds 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:00:38 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: SYNC 5000000005 done in 0.011 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000008 done in 0.011 seconds 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:00:38 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:00:38 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000010 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000014 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: SYNC 5000000009 done in 0.011 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000022 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: SYNC 5000000017 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000038 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_2: SYNC 5000000020 done in 0.002 seconds 2017-06-14 23:00:38 PDT INFO remoteWorkerThread_1: SYNC 5000000043 done in 0.002 seconds 2017-06-14 23:00:46 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:00:46 PDT INFO remoteWorkerThread_2: SYNC 5000000021 done in 0.003 seconds 2017-06-14 23:00:46 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:00:46 PDT INFO remoteWorkerThread_1: SYNC 5000000044 done in 0.017 seconds 2017-06-14 23:00:57 PDT INFO remoteWorkerThread_2: SYNC 5000000022 done in 0.002 seconds 2017-06-14 23:01:02 PDT INFO remoteWorkerThread_1: SYNC 5000000045 done in 0.005 seconds 2017-06-14 23:01:03 PDT INFO remoteWorkerThread_1: SYNC 5000000046 done in 0.004 seconds 2017-06-14 23:01:08 PDT INFO remoteWorkerThread_2: SYNC 5000000023 done in 0.003 seconds 2017-06-14 23:01:14 PDT INFO remoteWorkerThread_1: SYNC 5000000047 done in 0.004 seconds 2017-06-14 23:01:19 PDT INFO remoteWorkerThread_2: SYNC 5000000024 done in 0.003 seconds 2017-06-14 23:01:23 PDT INFO remoteWorkerThread_2: SYNC 5000000025 done in 0.003 seconds 2017-06-14 23:01:25 PDT INFO remoteWorkerThread_1: SYNC 5000000048 done in 0.004 seconds 2017-06-14 23:01:34 PDT INFO remoteWorkerThread_2: SYNC 5000000026 done in 0.002 seconds 2017-06-14 23:01:38 PDT INFO remoteWorkerThread_1: SYNC 5000000049 done in 0.004 seconds 2017-06-14 23:01:46 PDT INFO remoteWorkerThread_2: SYNC 5000000027 done in 0.003 seconds 2017-06-14 23:01:50 PDT INFO remoteWorkerThread_1: SYNC 5000000050 done in 0.005 seconds 2017-06-14 23:01:54 PDT INFO remoteWorkerThread_2: SYNC 5000000028 done in 0.002 seconds 2017-06-14 23:01:54 PDT INFO remoteWorkerThread_1: SYNC 5000000051 done in 0.005 seconds 2017-06-14 23:02:05 PDT INFO remoteWorkerThread_2: SYNC 5000000029 done in 0.003 seconds 2017-06-14 23:02:09 PDT INFO remoteWorkerThread_1: SYNC 5000000052 done in 0.005 seconds 2017-06-14 23:02:17 PDT INFO remoteWorkerThread_1: SYNC 5000000053 done in 0.005 seconds 2017-06-14 23:02:21 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:02:21 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:02:21 PDT INFO remoteWorkerThread_2: SYNC 5000000030 done in 0.027 seconds 2017-06-14 23:02:25 PDT INFO remoteWorkerThread_1: SYNC 5000000054 done in 0.007 seconds 2017-06-14 23:02:25 PDT INFO remoteWorkerThread_2: SYNC 5000000031 done in 0.008 seconds 2017-06-14 23:02:36 PDT INFO remoteWorkerThread_1: SYNC 5000000055 done in 0.008 seconds 2017-06-14 23:02:39 PDT INFO remoteWorkerThread_2: SYNC 5000000032 done in 0.005 seconds 2017-06-14 23:02:43 PDT INFO remoteWorkerThread_1: SYNC 5000000056 done in 0.005 seconds 2017-06-14 23:02:47 PDT INFO remoteWorkerThread_2: SYNC 5000000033 done in 0.005 seconds 2017-06-14 23:02:54 PDT INFO remoteWorkerThread_1: SYNC 5000000057 done in 0.004 seconds 2017-06-14 23:02:54 PDT INFO remoteWorkerThread_2: SYNC 5000000034 done in 0.005 seconds 2017-06-14 23:03:05 PDT INFO remoteWorkerThread_1: SYNC 5000000058 done in 0.005 seconds 2017-06-14 23:03:08 PDT INFO remoteWorkerThread_2: SYNC 5000000035 done in 0.005 seconds 2017-06-14 23:03:16 PDT CONFIG storeSet: set_id=1 set_origin=1 set_comment='Set 1 (il_longlive_schema) for region_il' 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_2: SYNC 5000000036 done in 0.005 seconds 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: connection for provider 2 terminated 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 2 2017-06-14 23:03:16 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:16 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000060 done in 0.012 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000062 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000064 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000066 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000068 done in 0.003 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000070 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000072 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000074 done in 0.002 seconds 2017-06-14 23:03:16 PDT INFO remoteWorkerThread_1: SYNC 5000000076 done in 0.002 seconds 2017-06-14 23:03:29 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:03:29 PDT INFO remoteWorkerThread_2: SYNC 5000000037 done in 0.005 seconds 2017-06-14 23:03:29 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:03:29 PDT INFO remoteWorkerThread_1: SYNC 5000000077 done in 0.015 seconds 2017-06-14 23:03:42 PDT INFO remoteWorkerThread_2: SYNC 5000000038 done in 0.013 seconds 2017-06-14 23:03:42 PDT INFO remoteWorkerThread_1: SYNC 5000000078 done in 0.015 seconds 2017-06-14 23:03:43 PDT INFO remoteWorkerThread_1: SYNC 5000000079 done in 0.004 seconds 2017-06-14 23:03:43 PDT INFO remoteWorkerThread_2: SYNC 5000000039 done in 0.005 seconds 2017-06-14 23:03:56 PDT INFO remoteWorkerThread_1: SYNC 5000000080 done in 0.004 seconds 2017-06-14 23:03:56 PDT INFO remoteWorkerThread_2: SYNC 5000000040 done in 0.005 seconds 2017-06-14 23:04:04 PDT INFO remoteWorkerThread_2: SYNC 5000000041 done in 0.005 seconds 2017-06-14 23:04:08 PDT INFO remoteWorkerThread_1: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:04:16 PDT INFO remoteWorkerThread_1: SYNC 5000000082 done in 0.005 seconds 2017-06-14 23:04:16 PDT INFO remoteWorkerThread_2: SYNC 5000000042 done in 0.006 seconds 2017-06-14 23:04:29 PDT INFO remoteWorkerThread_2: SYNC 5000000043 done in 0.004 seconds 2017-06-14 23:04:29 PDT INFO remoteWorkerThread_1: SYNC 5000000083 done in 0.005 seconds 2017-06-14 23:04:37 PDT INFO remoteWorkerThread_2: SYNC 5000000044 done in 0.004 seconds 2017-06-14 23:04:37 PDT INFO remoteWorkerThread_1: SYNC 5000000084 done in 0.005 seconds 2017-06-14 23:04:48 PDT INFO remoteWorkerThread_2: SYNC 5000000045 done in 0.004 seconds 2017-06-14 23:04:48 PDT INFO remoteWorkerThread_1: SYNC 5000000085 done in 0.005 seconds 2017-06-14 23:05:02 PDT INFO remoteWorkerThread_1: SYNC 5000000086 done in 0.004 seconds 2017-06-14 23:05:02 PDT INFO remoteWorkerThread_2: SYNC 5000000046 done in 0.005 seconds 2017-06-14 23:05:03 PDT INFO remoteWorkerThread_1: SYNC 5000000087 done in 0.005 seconds 2017-06-14 23:05:05 PDT INFO remoteWorkerThread_2: SYNC 5000000047 done in 0.005 seconds 2017-06-14 23:05:14 PDT INFO remoteWorkerThread_1: SYNC 5000000088 done in 0.005 seconds 2017-06-14 23:05:19 PDT INFO remoteWorkerThread_2: SYNC 5000000048 done in 0.004 seconds 2017-06-14 23:05:23 PDT INFO remoteWorkerThread_2: SYNC 5000000049 done in 0.005 seconds 2017-06-14 23:05:25 PDT INFO remoteWorkerThread_1: SYNC 5000000089 done in 0.004 seconds 2017-06-14 23:05:34 PDT INFO remoteWorkerThread_2: SYNC 5000000050 done in 0.006 seconds 2017-06-14 23:05:38 PDT INFO remoteWorkerThread_1: SYNC 5000000090 done in 0.005 seconds 2017-06-14 23:05:46 PDT INFO remoteWorkerThread_1: SYNC 5000000091 done in 0.004 seconds 2017-06-14 23:05:46 PDT INFO remoteWorkerThread_2: SYNC 5000000051 done in 0.005 seconds 2017-06-14 23:05:59 PDT INFO remoteWorkerThread_1: SYNC 5000000092 done in 0.004 seconds 2017-06-14 23:05:59 PDT INFO remoteWorkerThread_2: SYNC 5000000052 done in 0.005 seconds 2017-06-14 23:06:12 PDT INFO remoteWorkerThread_2: SYNC 5000000053 done in 0.004 seconds 2017-06-14 23:06:12 PDT INFO remoteWorkerThread_1: SYNC 5000000093 done in 0.004 seconds 2017-06-14 23:06:15 PDT INFO remoteWorkerThread_2: SYNC 5000000054 done in 0.004 seconds 2017-06-14 23:06:15 PDT INFO remoteWorkerThread_1: SYNC 5000000094 done in 0.005 seconds 2017-06-14 23:06:28 PDT INFO remoteWorkerThread_2: SYNC 5000000055 done in 0.005 seconds 2017-06-14 23:06:28 PDT INFO remoteWorkerThread_1: SYNC 5000000095 done in 0.005 seconds 2017-06-14 23:06:36 PDT INFO remoteWorkerThread_1: SYNC 5000000096 done in 0.004 seconds 2017-06-14 23:06:36 PDT INFO remoteWorkerThread_2: SYNC 5000000056 done in 0.005 seconds 2017-06-14 23:06:47 PDT INFO remoteWorkerThread_1: SYNC 5000000097 done in 0.005 seconds 2017-06-14 23:06:48 PDT INFO remoteWorkerThread_2: SYNC 5000000057 done in 0.004 seconds 2017-06-14 23:06:58 PDT INFO remoteWorkerThread_1: SYNC 5000000098 done in 0.004 seconds 2017-06-14 23:06:58 PDT INFO remoteWorkerThread_2: SYNC 5000000058 done in 0.005 seconds 2017-06-14 23:07:12 PDT INFO remoteWorkerThread_1: SYNC 5000000099 done in 0.004 seconds 2017-06-14 23:07:12 PDT INFO remoteWorkerThread_2: SYNC 5000000059 done in 0.005 seconds 2017-06-14 23:07:15 PDT INFO remoteWorkerThread_2: SYNC 5000000060 done in 0.004 seconds 2017-06-14 23:07:15 PDT INFO remoteWorkerThread_1: SYNC 5000000100 done in 0.005 seconds 2017-06-14 23:07:28 PDT INFO remoteWorkerThread_1: SYNC 5000000101 done in 0.060 seconds 2017-06-14 23:07:28 PDT INFO remoteWorkerThread_2: SYNC 5000000061 done in 0.062 seconds 2017-06-14 23:07:36 PDT INFO remoteWorkerThread_2: SYNC 5000000062 done in 0.004 seconds 2017-06-14 23:07:36 PDT INFO remoteWorkerThread_1: SYNC 5000000102 done in 0.005 seconds 2017-06-14 23:07:47 PDT INFO remoteWorkerThread_1: SYNC 5000000103 done in 0.008 seconds 2017-06-14 23:07:47 PDT INFO remoteWorkerThread_2: SYNC 5000000063 done in 0.009 seconds 2017-06-14 23:07:58 PDT INFO remoteWorkerThread_1: SYNC 5000000104 done in 0.005 seconds 2017-06-14 23:07:58 PDT INFO remoteWorkerThread_2: SYNC 5000000064 done in 0.005 seconds 2017-06-14 23:08:09 PDT INFO remoteWorkerThread_1: SYNC 5000000105 done in 0.007 seconds 2017-06-14 23:08:09 PDT INFO remoteWorkerThread_2: SYNC 5000000065 done in 0.009 seconds 2017-06-14 23:08:13 PDT INFO remoteWorkerThread_1: SYNC 5000000106 done in 0.005 seconds 2017-06-14 23:08:19 PDT INFO remoteWorkerThread_2: SYNC 5000000066 done in 0.010 seconds 2017-06-14 23:08:23 PDT INFO remoteWorkerThread_2: SYNC 5000000067 done in 0.004 seconds 2017-06-14 23:08:24 PDT INFO remoteWorkerThread_1: SYNC 5000000107 done in 0.008 seconds 2017-06-14 23:08:37 PDT INFO remoteWorkerThread_2: SYNC 5000000068 done in 0.006 seconds 2017-06-14 23:08:41 PDT INFO remoteWorkerThread_1: SYNC 5000000108 done in 0.005 seconds 2017-06-14 23:08:45 PDT INFO remoteWorkerThread_1: SYNC 5000000109 done in 0.005 seconds 2017-06-14 23:08:49 PDT INFO remoteWorkerThread_2: SYNC 5000000069 done in 0.005 seconds 2017-06-14 23:08:57 PDT INFO remoteWorkerThread_1: SYNC 5000000110 done in 0.006 seconds 2017-06-14 23:08:57 PDT INFO remoteWorkerThread_2: SYNC 5000000070 done in 0.006 seconds 2017-06-14 23:09:08 PDT INFO remoteWorkerThread_2: SYNC 5000000071 done in 0.005 seconds 2017-06-14 23:09:08 PDT INFO remoteWorkerThread_1: SYNC 5000000111 done in 0.005 seconds 2017-06-14 23:09:19 PDT INFO remoteWorkerThread_1: SYNC 5000000112 done in 0.004 seconds 2017-06-14 23:09:19 PDT INFO remoteWorkerThread_2: SYNC 5000000072 done in 0.005 seconds 2017-06-14 23:09:29 PDT INFO remoteWorkerThread_2: SYNC 5000000073 done in 0.004 seconds 2017-06-14 23:09:29 PDT INFO remoteWorkerThread_1: SYNC 5000000113 done in 0.005 seconds 2017-06-14 23:09:33 PDT INFO remoteWorkerThread_2: SYNC 5000000074 done in 0.005 seconds 2017-06-14 23:09:34 PDT INFO remoteWorkerThread_1: SYNC 5000000114 done in 0.005 seconds 2017-06-14 23:09:43 PDT INFO remoteWorkerThread_2: SYNC 5000000075 done in 0.005 seconds 2017-06-14 23:09:47 PDT INFO remoteWorkerThread_1: SYNC 5000000115 done in 0.004 seconds 2017-06-14 23:09:55 PDT INFO remoteWorkerThread_2: SYNC 5000000076 done in 0.005 seconds 2017-06-14 23:10:01 PDT INFO remoteWorkerThread_1: SYNC 5000000116 done in 0.005 seconds 2017-06-14 23:10:05 PDT INFO remoteWorkerThread_1: SYNC 5000000117 done in 0.005 seconds 2017-06-14 23:10:06 PDT INFO remoteWorkerThread_2: SYNC 5000000077 done in 0.005 seconds 2017-06-14 23:10:14 PDT INFO remoteWorkerThread_1: SYNC 5000000118 done in 0.005 seconds 2017-06-14 23:10:18 PDT INFO remoteWorkerThread_2: SYNC 5000000078 done in 0.005 seconds 2017-06-14 23:10:26 PDT INFO remoteWorkerThread_2: SYNC 5000000079 done in 0.005 seconds 2017-06-14 23:10:30 PDT INFO remoteWorkerThread_1: SYNC 5000000119 done in 0.005 seconds 2017-06-14 23:10:34 PDT INFO remoteWorkerThread_2: SYNC 5000000080 done in 0.005 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19256 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: Logswitch to sl_log_2 initiated CONTEXT: SQL statement "SELECT "_region_il".logswitch_start()" PL/pgSQL function _region_il.cleanupevent(interval) line 97 at PERFORM 2017-06-14 23:10:39 PDT INFO cleanupThread: 0.021 seconds for cleanupEvent() 2017-06-14 23:10:39 PDT INFO remoteWorkerThread_1: SYNC 5000000120 done in 0.005 seconds 2017-06-14 23:10:45 PDT INFO remoteWorkerThread_1: SYNC 5000000121 done in 0.005 seconds 2017-06-14 23:10:45 PDT INFO remoteWorkerThread_2: SYNC 5000000081 done in 0.005 seconds 2017-06-14 23:10:56 PDT INFO remoteWorkerThread_2: SYNC 5000000082 done in 0.005 seconds 2017-06-14 23:10:57 PDT INFO remoteWorkerThread_1: SYNC 5000000122 done in 0.004 seconds 2017-06-14 23:11:07 PDT INFO remoteWorkerThread_1: SYNC 5000000123 done in 0.005 seconds 2017-06-14 23:11:08 PDT INFO remoteWorkerThread_2: SYNC 5000000083 done in 0.004 seconds 2017-06-14 23:11:18 PDT INFO remoteWorkerThread_2: SYNC 5000000084 done in 0.005 seconds 2017-06-14 23:11:19 PDT INFO remoteWorkerThread_1: SYNC 5000000124 done in 0.004 seconds 2017-06-14 23:11:23 PDT INFO remoteWorkerThread_2: SYNC 5000000085 done in 0.004 seconds 2017-06-14 23:11:29 PDT INFO remoteWorkerThread_1: SYNC 5000000125 done in 0.005 seconds 2017-06-14 23:11:34 PDT INFO remoteWorkerThread_1: SYNC 5000000126 done in 0.005 seconds 2017-06-14 23:11:34 PDT INFO remoteWorkerThread_2: SYNC 5000000086 done in 0.005 seconds 2017-06-14 23:11:45 PDT INFO remoteWorkerThread_1: SYNC 5000000127 done in 0.004 seconds 2017-06-14 23:11:45 PDT INFO remoteWorkerThread_2: SYNC 5000000087 done in 0.004 seconds 2017-06-14 23:11:56 PDT INFO remoteWorkerThread_2: SYNC 5000000088 done in 0.005 seconds 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:11:56 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: connection for provider 2 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 2 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_2: connection for provider 1 terminated 2017-06-14 23:11:56 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 1 2017-06-14 23:12:00 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:12:00 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:12:00 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:12:00 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:12:00 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:12:00 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:12:00 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:12:00 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:00 PDT INFO remoteWorkerThread_1: SYNC 5000000130 done in 0.012 seconds 2017-06-14 23:12:08 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:12:08 PDT INFO remoteWorkerThread_1: SYNC 5000000131 done in 0.003 seconds 2017-06-14 23:12:08 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:08 PDT INFO remoteWorkerThread_2: SYNC 5000000089 done in 0.014 seconds 2017-06-14 23:12:19 PDT INFO remoteWorkerThread_1: SYNC 5000000132 done in 0.002 seconds 2017-06-14 23:12:30 PDT INFO remoteWorkerThread_1: SYNC 5000000133 done in 0.003 seconds 2017-06-14 23:12:34 PDT INFO remoteWorkerThread_1: SYNC 5000000134 done in 0.003 seconds 2017-06-14 23:12:34 PDT INFO remoteWorkerThread_2: SYNC 5000000090 done in 0.003 seconds 2017-06-14 23:12:38 PDT INFO remoteWorkerThread_2: SYNC 5000000091 done in 0.003 seconds 2017-06-14 23:12:49 PDT INFO remoteWorkerThread_1: SYNC 5000000135 done in 0.002 seconds 2017-06-14 23:12:53 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:12:53 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:12:56 PDT INFO remoteWorkerThread_1: SYNC 5000000136 done in 0.002 seconds 2017-06-14 23:13:02 PDT INFO remoteWorkerThread_2: SYNC 5000000092 done in 8.409 seconds 2017-06-14 23:13:07 PDT INFO remoteWorkerThread_1: SYNC 5000000137 done in 0.003 seconds 2017-06-14 23:13:07 PDT INFO remoteWorkerThread_2: SYNC 5000000093 done in 0.005 seconds 2017-06-14 23:13:18 PDT INFO remoteWorkerThread_1: SYNC 5000000138 done in 0.003 seconds 2017-06-14 23:13:18 PDT INFO remoteWorkerThread_2: SYNC 5000000094 done in 0.005 seconds 2017-06-14 23:13:29 PDT INFO remoteWorkerThread_1: SYNC 5000000139 done in 0.003 seconds 2017-06-14 23:13:29 PDT INFO remoteWorkerThread_2: SYNC 5000000095 done in 0.005 seconds 2017-06-14 23:13:40 PDT INFO remoteWorkerThread_1: SYNC 5000000140 done in 0.003 seconds 2017-06-14 23:13:40 PDT INFO remoteWorkerThread_2: SYNC 5000000096 done in 0.006 seconds 2017-06-14 23:13:44 PDT INFO remoteWorkerThread_1: SYNC 5000000141 done in 0.003 seconds 2017-06-14 23:13:48 PDT INFO remoteWorkerThread_2: SYNC 5000000097 done in 0.006 seconds 2017-06-14 23:13:58 PDT INFO remoteWorkerThread_2: SYNC 5000000098 done in 0.005 seconds 2017-06-14 23:14:02 PDT INFO remoteWorkerThread_1: SYNC 5000000142 done in 0.003 seconds 2017-06-14 23:14:06 PDT INFO remoteWorkerThread_1: SYNC 5000000143 done in 0.002 seconds 2017-06-14 23:14:16 PDT INFO remoteWorkerThread_2: SYNC 5000000099 done in 0.005 seconds 2017-06-14 23:14:20 PDT INFO remoteWorkerThread_2: SYNC 5000000100 done in 0.007 seconds 2017-06-14 23:14:20 PDT INFO remoteWorkerThread_1: SYNC 5000000144 done in 0.003 seconds 2017-06-14 23:14:28 PDT INFO remoteWorkerThread_1: SYNC 5000000145 done in 0.003 seconds 2017-06-14 23:14:34 PDT INFO remoteWorkerThread_2: SYNC 5000000101 done in 0.015 seconds 2017-06-14 23:14:39 PDT INFO remoteWorkerThread_1: SYNC 5000000146 done in 0.003 seconds 2017-06-14 23:14:50 PDT INFO remoteWorkerThread_1: SYNC 5000000147 done in 0.003 seconds 2017-06-14 23:14:51 PDT CONFIG storeSubscribe: sub_set=1 sub_provider=2 sub_forward='t' 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: connection for provider 1 terminated 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 1 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:14:51 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=1 2017-06-14 23:14:51 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:14:51 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:14:51 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_1: connection for provider 1 terminated 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_1: disconnecting from data provider 1 2017-06-14 23:14:51 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:51 PDT WARN remoteWorkerThread_1: copy set: data provider 2 only on sync 5000000128 - sleep 5 seconds 2017-06-14 23:14:55 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:14:55 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:55 PDT INFO remoteWorkerThread_2: SYNC 5000000102 done in 0.012 seconds 2017-06-14 23:14:56 PDT INFO copy_set 1 - omit=f - bool=0 2017-06-14 23:14:56 PDT INFO omit is FALSE 2017-06-14 23:14:56 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: connected to provider DB 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."entity" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."entity_result" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."result" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: prepare to copy table "il_longlive"."result_logbook" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: all tables for set 1 found on subscriber 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."entity_entity_id_seq" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."entity_result_id_seq" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."result_id_seq" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: copy sequence "il_longlive"."result_logbook_record_id_seq" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."entity" 2017-06-14 23:14:56 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."entity" NOTICE: truncate of "il_longlive"."entity" succeeded 2017-06-14 23:15:03 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:15:03 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:15:03 PDT INFO remoteWorkerThread_2: SYNC 5000000103 done in 0.033 seconds 2017-06-14 23:15:17 PDT CONFIG remoteWorkerThread_1: 125045870 bytes copied for table "il_longlive"."entity" 2017-06-14 23:15:19 PDT INFO remoteWorkerThread_2: SYNC 5000000104 done in 0.006 seconds 2017-06-14 23:15:22 PDT INFO remoteWorkerThread_2: SYNC 5000000105 done in 0.005 seconds 2017-06-14 23:15:25 PDT CONFIG remoteWorkerThread_1: 28.669 seconds to copy table "il_longlive"."entity" 2017-06-14 23:15:25 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."entity_result" 2017-06-14 23:15:25 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."entity_result" NOTICE: truncate of "il_longlive"."entity_result" succeeded 2017-06-14 23:15:34 PDT INFO remoteWorkerThread_2: SYNC 5000000106 done in 0.004 seconds 2017-06-14 23:15:37 PDT CONFIG remoteWorkerThread_1: 66996342 bytes copied for table "il_longlive"."entity_result" 2017-06-14 23:15:45 PDT INFO remoteWorkerThread_2: SYNC 5000000107 done in 0.012 seconds 2017-06-14 23:15:56 PDT INFO remoteWorkerThread_2: SYNC 5000000108 done in 0.004 seconds 2017-06-14 23:16:07 PDT INFO remoteWorkerThread_2: SYNC 5000000109 done in 0.005 seconds 2017-06-14 23:16:10 PDT CONFIG remoteWorkerThread_1: 45.106 seconds to copy table "il_longlive"."entity_result" 2017-06-14 23:16:10 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."result" 2017-06-14 23:16:10 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."result" NOTICE: truncate of "il_longlive"."result" succeeded 2017-06-14 23:16:11 PDT INFO remoteWorkerThread_2: SYNC 5000000110 done in 0.015 seconds 2017-06-14 23:16:27 PDT CONFIG remoteWorkerThread_1: 105961503 bytes copied for table "il_longlive"."result" 2017-06-14 23:16:29 PDT INFO remoteWorkerThread_2: SYNC 5000000111 done in 0.005 seconds 2017-06-14 23:16:32 PDT INFO remoteWorkerThread_2: SYNC 5000000112 done in 0.005 seconds 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: 25.058 seconds to copy table "il_longlive"."result" 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: copy table "il_longlive"."result_logbook" 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: Begin COPY of table "il_longlive"."result_logbook" NOTICE: truncate of "il_longlive"."result_logbook" succeeded 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: 79 bytes copied for table "il_longlive"."result_logbook" 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: 0.079 seconds to copy table "il_longlive"."result_logbook" 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 101 ("il_longlive"."entity_entity_id_seq") to 2528612 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 103 ("il_longlive"."result_id_seq") to 2611527 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 102 ("il_longlive"."entity_result_id_seq") to 2611527 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: set last_value of sequence 104 ("il_longlive"."result_logbook_record_id_seq") to 1 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: 0.002 seconds to copy sequences 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: copy_set SYNC found, use event seqno 5000000147. 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: 0.015 seconds to build initial setsync status 2017-06-14 23:16:35 PDT INFO copy_set 1 done in 98.961 seconds 2017-06-14 23:16:35 PDT CONFIG enableSubscription: sub_set=1 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: connection for provider 2 terminated 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 2 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: connection for provider 1 terminated 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: disconnecting from data provider 1 2017-06-14 23:16:35 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=2 2017-06-14 23:16:35 PDT CONFIG storeListen: li_origin=2 li_receiver=3 li_provider=1 2017-06-14 23:16:35 PDT CONFIG storeListen: li_origin=1 li_receiver=3 li_provider=2 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 2 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: update provider configuration 2017-06-14 23:16:35 PDT CONFIG remoteWorkerThread_1: added active set 1 to provider 2 2017-06-14 23:16:35 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:16:35 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: SYNC 5000000151 done in 0.042 seconds 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: SYNC 5000000155 done in 0.006 seconds 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:35 PDT INFO remoteWorkerThread_1: SYNC 5000000159 done in 0.005 seconds 2017-06-14 23:16:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:37 PDT INFO remoteWorkerThread_1: SYNC 5000000160 done in 0.008 seconds 2017-06-14 23:16:41 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:16:41 PDT CONFIG version for "host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:16:41 PDT INFO remoteWorkerThread_2: SYNC 5000000113 done in 0.013 seconds 2017-06-14 23:16:55 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:55 PDT INFO remoteWorkerThread_2: SYNC 5000000114 done in 0.005 seconds 2017-06-14 23:16:55 PDT INFO remoteWorkerThread_1: SYNC 5000000161 done in 0.008 seconds 2017-06-14 23:16:59 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:16:59 PDT INFO remoteWorkerThread_1: SYNC 5000000162 done in 0.007 seconds 2017-06-14 23:17:03 PDT INFO remoteWorkerThread_2: SYNC 5000000115 done in 0.003 seconds 2017-06-14 23:17:14 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:17:14 PDT INFO remoteWorkerThread_2: SYNC 5000000116 done in 0.004 seconds 2017-06-14 23:17:14 PDT INFO remoteWorkerThread_1: SYNC 5000000163 done in 0.008 seconds 2017-06-14 23:17:28 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:17:28 PDT INFO remoteWorkerThread_2: SYNC 5000000117 done in 0.004 seconds 2017-06-14 23:17:28 PDT INFO remoteWorkerThread_1: SYNC 5000000165 done in 0.008 seconds 2017-06-14 23:17:32 PDT INFO remoteWorkerThread_2: SYNC 5000000118 done in 0.003 seconds 2017-06-14 23:17:36 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:17:36 PDT INFO remoteWorkerThread_1: SYNC 5000000166 done in 0.007 seconds 2017-06-14 23:17:47 PDT INFO remoteWorkerThread_2: SYNC 5000000119 done in 0.002 seconds 2017-06-14 23:17:48 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:17:48 PDT INFO remoteWorkerThread_1: SYNC 5000000167 done in 0.007 seconds 2017-06-14 23:17:52 PDT INFO remoteWorkerThread_2: SYNC 5000000120 done in 0.003 seconds 2017-06-14 23:18:03 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:03 PDT INFO remoteWorkerThread_2: SYNC 5000000121 done in 0.004 seconds 2017-06-14 23:18:03 PDT INFO remoteWorkerThread_1: SYNC 5000000168 done in 0.008 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_2: SYNC 5000000122 done in 0.004 seconds 2017-06-14 23:18:14 PDT INFO remoteWorkerThread_1: SYNC 5000000169 done in 0.007 seconds 2017-06-14 23:18:18 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:18 PDT INFO remoteWorkerThread_1: SYNC 5000000170 done in 0.022 seconds 2017-06-14 23:18:26 PDT INFO remoteWorkerThread_2: SYNC 5000000123 done in 0.003 seconds 2017-06-14 23:18:30 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:30 PDT INFO remoteWorkerThread_1: SYNC 5000000171 done in 0.007 seconds 2017-06-14 23:18:31 PDT INFO remoteWorkerThread_2: SYNC 5000000124 done in 0.002 seconds 2017-06-14 23:18:44 PDT CONFIG remoteWorkerThread_2: update provider configuration 2017-06-14 23:18:44 PDT CONFIG version for "host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888" is 90505 2017-06-14 23:18:44 PDT INFO remoteWorkerThread_2: SYNC 5000000125 done in 0.015 seconds 2017-06-14 23:18:49 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:49 PDT INFO remoteWorkerThread_1: SYNC 5000000172 done in 0.007 seconds 2017-06-14 23:18:53 PDT INFO remoteWorkerThread_2: SYNC 5000000126 done in 0.004 seconds 2017-06-14 23:18:54 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:54 PDT INFO remoteWorkerThread_1: SYNC 5000000173 done in 0.006 seconds 2017-06-14 23:18:58 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:18:58 PDT INFO remoteWorkerThread_1: SYNC 5000000174 done in 0.006 seconds 2017-06-14 23:19:02 PDT INFO remoteWorkerThread_2: SYNC 5000000127 done in 0.012 seconds 2017-06-14 23:19:10 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:19:10 PDT INFO remoteWorkerThread_1: SYNC 5000000175 done in 0.015 seconds 2017-06-14 23:19:11 PDT INFO remoteWorkerThread_2: SYNC 5000000128 done in 0.004 seconds 2017-06-14 23:19:22 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:19:22 PDT INFO remoteWorkerThread_2: SYNC 5000000129 done in 0.005 seconds 2017-06-14 23:19:22 PDT INFO remoteWorkerThread_1: SYNC 5000000176 done in 0.007 seconds 2017-06-14 23:19:36 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:19:36 PDT INFO remoteWorkerThread_2: SYNC 5000000130 done in 0.006 seconds 2017-06-14 23:19:36 PDT INFO remoteWorkerThread_1: SYNC 5000000177 done in 0.007 seconds 2017-06-14 23:19:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:19:37 PDT INFO remoteWorkerThread_1: SYNC 5000000178 done in 0.006 seconds 2017-06-14 23:19:41 PDT INFO remoteWorkerThread_2: SYNC 5000000131 done in 0.005 seconds 2017-06-14 23:19:55 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:19:55 PDT INFO remoteWorkerThread_2: SYNC 5000000132 done in 0.006 seconds 2017-06-14 23:19:55 PDT INFO remoteWorkerThread_1: SYNC 5000000179 done in 0.007 seconds 2017-06-14 23:20:06 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:20:06 PDT INFO remoteWorkerThread_2: SYNC 5000000133 done in 0.005 seconds 2017-06-14 23:20:06 PDT INFO remoteWorkerThread_1: SYNC 5000000180 done in 0.007 seconds 2017-06-14 23:20:16 PDT INFO remoteWorkerThread_2: SYNC 5000000134 done in 0.007 seconds 2017-06-14 23:20:20 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:20:20 PDT INFO remoteWorkerThread_1: SYNC 5000000182 done in 0.007 seconds 2017-06-14 23:20:21 PDT INFO remoteWorkerThread_2: SYNC 5000000135 done in 0.005 seconds 2017-06-14 23:20:35 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:20:35 PDT INFO remoteWorkerThread_2: SYNC 5000000136 done in 0.006 seconds 2017-06-14 23:20:35 PDT INFO remoteWorkerThread_1: SYNC 5000000183 done in 0.008 seconds NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19234 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19239 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19244 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19348 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19566 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19567 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19568 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: cleanup stale sl_nodelock entry for pid=19663 CONTEXT: SQL statement "SELECT "_region_il".cleanupNodelock()" PL/pgSQL function _region_il.cleanupevent(interval) line 82 at PERFORM NOTICE: Slony-I: log switch to sl_log_2 complete - truncate sl_log_1 CONTEXT: PL/pgSQL function _region_il.cleanupevent(interval) line 95 at assignment 2017-06-14 23:20:39 PDT INFO cleanupThread: 0.077 seconds for cleanupEvent() 2017-06-14 23:20:39 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:20:39 PDT INFO remoteWorkerThread_1: SYNC 5000000184 done in 0.008 seconds 2017-06-14 23:20:43 PDT INFO remoteWorkerThread_2: SYNC 5000000137 done in 0.005 seconds 2017-06-14 23:20:54 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:20:54 PDT INFO remoteWorkerThread_2: SYNC 5000000138 done in 0.005 seconds 2017-06-14 23:20:54 PDT INFO remoteWorkerThread_1: SYNC 5000000185 done in 0.007 seconds 2017-06-14 23:21:04 PDT INFO remoteWorkerThread_2: SYNC 5000000139 done in 0.009 seconds 2017-06-14 23:21:08 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:21:08 PDT INFO remoteWorkerThread_1: SYNC 5000000186 done in 0.006 seconds 2017-06-14 23:21:11 PDT INFO remoteWorkerThread_2: SYNC 5000000140 done in 0.005 seconds 2017-06-14 23:21:16 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:21:16 PDT INFO remoteWorkerThread_1: SYNC 5000000188 done in 0.007 seconds 2017-06-14 23:21:27 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:21:27 PDT INFO remoteWorkerThread_2: SYNC 5000000141 done in 0.006 seconds 2017-06-14 23:21:27 PDT INFO remoteWorkerThread_1: SYNC 5000000189 done in 0.007 seconds 2017-06-14 23:21:31 PDT INFO remoteWorkerThread_2: SYNC 5000000142 done in 0.005 seconds 2017-06-14 23:21:45 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:21:45 PDT INFO remoteWorkerThread_2: SYNC 5000000143 done in 0.005 seconds 2017-06-14 23:21:45 PDT INFO remoteWorkerThread_1: SYNC 5000000190 done in 0.007 seconds 2017-06-14 23:21:49 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:21:49 PDT INFO remoteWorkerThread_1: SYNC 5000000191 done in 0.008 seconds 2017-06-14 23:21:52 PDT INFO remoteWorkerThread_2: SYNC 5000000144 done in 0.005 seconds 2017-06-14 23:22:05 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:22:05 PDT INFO remoteWorkerThread_2: SYNC 5000000145 done in 0.006 seconds 2017-06-14 23:22:05 PDT INFO remoteWorkerThread_1: SYNC 5000000192 done in 0.008 seconds 2017-06-14 23:22:18 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:22:18 PDT INFO remoteWorkerThread_2: SYNC 5000000146 done in 0.005 seconds 2017-06-14 23:22:18 PDT INFO remoteWorkerThread_1: SYNC 5000000194 done in 0.007 seconds 2017-06-14 23:22:22 PDT INFO remoteWorkerThread_2: SYNC 5000000147 done in 0.005 seconds 2017-06-14 23:22:30 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:22:30 PDT INFO remoteWorkerThread_1: SYNC 5000000195 done in 0.016 seconds 2017-06-14 23:22:31 PDT INFO remoteWorkerThread_2: SYNC 5000000148 done in 0.005 seconds 2017-06-14 23:22:45 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:22:45 PDT INFO remoteWorkerThread_2: SYNC 5000000149 done in 0.006 seconds 2017-06-14 23:22:45 PDT INFO remoteWorkerThread_1: SYNC 5000000196 done in 0.008 seconds 2017-06-14 23:22:56 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:22:56 PDT INFO remoteWorkerThread_2: SYNC 5000000150 done in 0.006 seconds 2017-06-14 23:22:56 PDT INFO remoteWorkerThread_1: SYNC 5000000198 done in 0.008 seconds 2017-06-14 23:23:10 PDT INFO remoteWorkerThread_2: SYNC 5000000151 done in 0.005 seconds 2017-06-14 23:23:13 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:23:13 PDT INFO remoteWorkerThread_2: SYNC 5000000152 done in 0.017 seconds 2017-06-14 23:23:13 PDT INFO remoteWorkerThread_1: SYNC 5000000199 done in 0.019 seconds 2017-06-14 23:23:26 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:23:26 PDT INFO remoteWorkerThread_2: SYNC 5000000153 done in 0.005 seconds 2017-06-14 23:23:26 PDT INFO remoteWorkerThread_1: SYNC 5000000201 done in 0.007 seconds 2017-06-14 23:23:39 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:23:39 PDT INFO remoteWorkerThread_2: SYNC 5000000154 done in 0.006 seconds 2017-06-14 23:23:39 PDT INFO remoteWorkerThread_1: SYNC 5000000202 done in 0.007 seconds 2017-06-14 23:23:43 PDT INFO remoteWorkerThread_2: SYNC 5000000155 done in 0.004 seconds 2017-06-14 23:23:51 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:23:51 PDT INFO remoteWorkerThread_2: SYNC 5000000156 done in 0.005 seconds 2017-06-14 23:23:51 PDT INFO remoteWorkerThread_1: SYNC 5000000203 done in 0.007 seconds 2017-06-14 23:23:59 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:23:59 PDT INFO remoteWorkerThread_1: SYNC 5000000204 done in 0.007 seconds 2017-06-14 23:24:07 PDT INFO remoteWorkerThread_2: SYNC 5000000157 done in 0.005 seconds 2017-06-14 23:24:10 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:24:10 PDT INFO remoteWorkerThread_1: SYNC 5000000205 done in 0.008 seconds 2017-06-14 23:24:13 PDT INFO remoteWorkerThread_2: SYNC 5000000158 done in 0.005 seconds 2017-06-14 23:24:16 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:24:16 PDT INFO remoteWorkerThread_1: SYNC 5000000206 done in 0.006 seconds 2017-06-14 23:24:24 PDT INFO remoteWorkerThread_2: SYNC 5000000159 done in 0.005 seconds 2017-06-14 23:24:37 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:24:37 PDT INFO remoteWorkerThread_2: SYNC 5000000160 done in 0.006 seconds 2017-06-14 23:24:37 PDT INFO remoteWorkerThread_1: SYNC 5000000207 done in 0.008 seconds 2017-06-14 23:24:50 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:24:50 PDT INFO remoteWorkerThread_2: SYNC 5000000161 done in 0.005 seconds 2017-06-14 23:24:50 PDT INFO remoteWorkerThread_1: SYNC 5000000209 done in 0.007 seconds 2017-06-14 23:24:53 PDT INFO remoteWorkerThread_2: SYNC 5000000162 done in 0.005 seconds 2017-06-14 23:25:06 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:25:06 PDT INFO remoteWorkerThread_2: SYNC 5000000163 done in 0.013 seconds 2017-06-14 23:25:06 PDT INFO remoteWorkerThread_1: SYNC 5000000210 done in 0.014 seconds 2017-06-14 23:25:19 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:25:19 PDT INFO remoteWorkerThread_2: SYNC 5000000164 done in 0.005 seconds 2017-06-14 23:25:19 PDT INFO remoteWorkerThread_1: SYNC 5000000211 done in 0.007 seconds 2017-06-14 23:25:22 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:25:22 PDT INFO remoteWorkerThread_2: SYNC 5000000165 done in 0.006 seconds 2017-06-14 23:25:22 PDT INFO remoteWorkerThread_1: SYNC 5000000212 done in 0.008 seconds 2017-06-14 23:25:30 PDT INFO remoteWorkerThread_1: syncing set 1 with 4 table(s) from provider 2 2017-06-14 23:25:30 PDT INFO remoteWorkerThread_1: SYNC 5000000213 done in 0.006 seconds 2017-06-14 23:25:31 PDT INFO remoteWorkerThread_2: SYNC 5000000166 done in 0.004 seconds13. 查看数据同步状态
$ psql -h 10.189.100.226 -U slonyadmin -d region_il -c "\d il_longlive.entity" -W Password for user slonyadmin: Table "il_longlive.entity" Column | Type | Modifiers --------------+------------------------+------------------------------------------------------------------------ entity_id | integer | not null default nextval('il_longlive.entity_entity_id_seq'::regclass) country_code | character varying(3) | state_code | character varying(50) | name | character varying(128) | islive | boolean | Indexes: "entity_pkey" PRIMARY KEY, btree (entity_id) Triggers: #可以看到Slony-I配置完成后原表多了触发器 _region_il_denyaccess BEFORE INSERT OR DELETE OR UPDATE ON il_longlive.entity FOR EACH ROW EXECUTE PROCEDURE _region_il.denyaccess('_region_il') _region_il_truncatedeny BEFORE TRUNCATE ON il_longlive.entity FOR EACH STATEMENT EXECUTE PROCEDURE _region_il.deny_truncate() Disabled user triggers: _region_il_logtrigger AFTER INSERT OR DELETE OR UPDATE ON il_longlive.entity FOR EACH ROW EXECUTE PROCEDURE _region_il.logtrigger('_region_il', '101', 'k') _region_il_truncatetrigger BEFORE TRUNCATE ON il_longlive.entity FOR EACH STATEMENT EXECUTE PROCEDURE _region_il.log_truncate('101') $ psql -h 10.189.102.118 -U slonyadmin -d region_il -c "select count(*) from il_longlive.entity;" -W Password for user slonyadmin: count --------- (1 row) $ psql -h 10.189.100.195 -U slonyadmin -d region_il -c "select count(*) from il_longlive.entity;" -W Password for user slonyadmin: count --------- #可以看到数据已经同步到辅节点1了 (1 row) $ psql -h 10.189.100.226 -U slonyadmin -d region_il -c "select count(*) from il_longlive.entity;" -W Password for user slonyadmin: count --------- #可以看到数据已经同步到辅节点2了 (1 row)14. Slony-I相关表或者视图查看
sl_apply_stats: 每个节点本地的复制表的DML统计状态
sl_components: slon/slonik组件等当前的工作状态
region_il=# select * from sl_components; co_actor | co_pid | co_node | co_connection_pid | co_activity | co_starttime | co_event | co_eventtype ----------------------+--------+---------+-------------------+------------------+------------------------+------------+-------------- local_monitor | 2455 | 0 | 2470 | thread main loop | 2017-06-14 22:54:42-07 | | n/a local_cleanup | 2455 | 0 | 2468 | cleanupEvent | 2017-06-15 00:44:48-07 | | n/a remoteWorkerThread_2 | 2455 | 2 | 2471 | SYNC | 2017-06-15 00:49:14-07 | 5000000668 | SYNC local_listen | 2455 | 1 | 2459 | thread main loop | 2017-06-15 00:49:20-07 | | n/a remoteWorkerThread_3 | 2455 | 3 | 2467 | SYNC | 2017-06-15 00:49:17-07 | 5000000653 | SYNC remote listener | 2455 | 3 | 19059 | thread main loop | 2017-06-15 00:49:18-07 | | n/a local_sync | 2455 | 0 | 2469 | thread main loop | 2017-06-15 00:49:21-07 | | n/a (7 rows)sl_table: 当前集群中的表信息
region_il=# select * from sl_table ; tab_id | tab_reloid | tab_relname | tab_nspname | tab_set | tab_idxname | tab_altered | tab_comment --------+------------+----------------+-------------+---------+---------------------+-------------+--------------------------------------------------- | 19350 | entity | il_longlive | 1 | entity_pkey | f | Table il_longlive.entity with primary key | 19355 | entity_result | il_longlive | 1 | entity_result_pkey | f | Table il_longlive.entity_result with primary key | 19360 | result | il_longlive | 1 | result_pkey | f | Table il_longlive.result with primary key | 19365 | result_logbook | il_longlive | 1 | result_logbook_pkey | f | Table il_longlive.result_logbook with primary key (4 rows)sl_set: 当前集群中的集合信息
region_il=# select * from sl_set; set_id | set_origin | set_locked | set_comment --------+------------+------------+------------------------------------------ 1 | 1 | | Set 1 (il_longlive_schema) for region_il (1 row)sl_set: 当前集群中的节点信息
region_il=# select * from sl_node; no_id | no_active | no_comment | no_failed -------+-----------+-----------------------------------+----------- | t | Node 1 - region_il@10.189.102.118 | f | t | Node 2 - region_il@10.189.100.195 | f | t | Node 3 - region_il@10.189.100.226 | f (3 rows)sl_status: 显示远程节点落后主节点的延迟
region_il=# select * from sl_status; -[ RECORD 1 ]-------------+------------------------------ st_origin | 1 st_received | 3 st_last_event | 5000000800 st_last_event_ts | 2017-06-15 01:03:19.585161-07 st_last_received | 5000000799 st_last_received_ts | 2017-06-15 01:03:14.281766-07 st_last_received_event_ts | 2017-06-15 01:03:09.57357-07 st_lag_num_events | 1 st_lag_time | 00:00:11.414753 -[ RECORD 2 ]-------------+------------------------------ st_origin | 1 st_received | 2 st_last_event | 5000000800 st_last_event_ts | 2017-06-15 01:03:19.585161-07 st_last_received | 5000000799 st_last_received_ts | 2017-06-15 01:03:14.193743-07 st_last_received_event_ts | 2017-06-15 01:03:09.57357-07 st_lag_num_events | 1 st_lag_time | 00:00:11.414753sl_subscribe: 显示当前集群中的订阅者
region_il=# select * from sl_subscribe; sub_set | sub_provider | sub_receiver | sub_forward | sub_active ---------+--------------+--------------+-------------+------------ | 1 | 2 | t | t | 2 | 3 | t | t (2 rows)Slony-I日常维护
1. 更改主表的表结构结构
1.1. 查看要修改的表结构信息
region_il=# \d result_logbook; Table "il_longlive.result_logbook" Column | Type | Modifiers ------------------+-----------------------------+-------------------------------------------------------------------- record_id | integer | not null default nextval('result_logbook_record_id_seq'::regclass) schema | character varying(255) | not null vendor_code | character varying(255) | tag | character varying(255) | total_result_cnt | integer | exist_result_cnt | integer | new_result_cnt | integer | max_result_id | integer | islive | boolean | default true create_time | timestamp without time zone | default now() update_time | timestamp without time zone | Indexes: "result_logbook_pkey" PRIMARY KEY, btree (record_id)1.2. 创建修改表字段的ddl语句文件
$ cat /home/postgres/alter_table_result_logbook.sql alter table il_longlive.result_logbook add column address varchar(40);1.3. 通过执行slonik_execute_script命令
$ /usr/local/pgsql/bin/slonik_execute_script 1 /home/postgres/alter_table_result_logbook.sql cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; execute script ( set id = 1, filename = '/home/postgres/alter_table_result_logbook.sql', event node = 1 ); $ cat /home/postgres/alter_table_result_logbook.sh #!/bin/sh slonik << __EOF__ cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; execute script ( filename = '/home/postgres/alter_table_result_logbook.sql', event node = 1 ); __EOF__ $ chmod +x /home/postgres/alter_table_result_logbook.sh $ /home/postgres/alter_table_result_logbook.sh1.4. 验证所有节点上的表结构修改是否成功
$ psql -h 10.189.102.118 -U slonyadmin -d region_il -c "\d il_longlive.result_logbook" | grep address Password for user slonyadmin: address | character varying(40) $ psql -h 10.189.100.195 -U slonyadmin -d region_il -c "\d il_longlive.result_logbook" | grep address Password for user slonyadmin: address | character varying(40) $ psql -h 10.189.100.226 -U slonyadmin -d region_il -c "\d il_longlive.result_logbook" | grep address Password for user slonyadmin: address | character varying(40)2. 向现有集群中增加一个复制表
2.1. 查看主库的的表结构信息,确认符合Slony-I的要求(主键)
region_il=# \d snapshot Table "il_longlive.snapshot" Column | Type | Modifiers -----------------+--------------------------------+------------------------------------------------------- id | integer | not null default nextval('snapshot_id_seq'::regclass) feed_schema | character varying(255) | job | character varying(255) | table_name | character varying(255) | max_primary_key | bigint | create_time | timestamp(0) without time zone | update_time | timestamp(0) without time zone | Indexes: "snapshot_pkey" PRIMARY KEY, btree (id)2.2. 将主表结构导入到所有辅节点
$ pg_dump -U slonyadmin -s -t il_longlive.snapshot region_il | psql -h 10.189.100.195 -U slonyadmin region_il $ pg_dump -U slonyadmin -s -t il_longlive.snapshot region_il | psql -h 10.189.100.226 -U slonyadmin region_il2.3. 修改Slony-I配置文件slon_tools_conf,增加新复制集,并复制到所有节点(不是必须,为了各节点配置文件完整性)
$ cat /usr/local/pgsql/etc/slon_tools.conf if ($ENV{"SLONYNODES"}) { require $ENV{"SLONYNODES"}; } else { $CLUSTER_NAME = 'region_il'; $PIDFILE_DIR = '/usr/local/pgsql/data'; $LOGDIR = '/usr/local/pgsql/data/pg_log'; $MASTERNODE = 1; $DEBUGLEVEL = 0; $START_WATCHDOG = 1; $SLEEP_TIME = 60; $WATCHDOG_VERBOSE = 0; add_node(node => 1, host => '10.189.102.118', dbname => 'region_il', port => 5432, user => 'slonyadmin', password => '888888',); add_node(node => 2, parent => 1, host => '10.189.100.195', dbname => 'region_il', port => 5432, user => 'slonyadmin', password => '888888',); add_node(node => 3, parent => 2, host => '10.189.100.226', dbname => 'region_il', port => 5432, user => 'slonyadmin', password => '888888',); } $SLONY_SETS = { "il_longlive_schema" => { "set_id" => 1, "table_id" => 101, "sequence_id" => 101, "pkeyedtables" => [ 'il_longlive.entity', 'il_longlive.entity_result', 'il_longlive.result', 'il_longlive.result_logbook', ], "keyedtables" => {}, "sequences" => ['il_longlive.entity_entity_id_seq', 'il_longlive.entity_result_id_seq', 'il_longlive.result_id_seq', 'il_longlive.result_logbook_record_id_seq', ], }, "il_longlive_schema1" => { "set_id" => 2, "table_id" => 201, "sequence_id" => 201, "pkeyedtables" => [ 'il_longlive.snapshot', ], "keyedtables" => {}, "sequences" => ['il_longlive.snapshot_id_seq' ], }, }; if ($ENV{"SLONYSET"}) { require $ENV{"SLONYSET"}; } # Please do not add or change anything below this point. 1; $ scp /usr/local/pgsql/etc/slon_tools.conf postgres@10.189.100.195:/usr/local/pgsql/etc/slon_tools.conf $ scp /usr/local/pgsql/etc/slon_tools.conf postgres@10.189.100.226:/usr/local/pgsql/etc/slon_tools.conf2.4. 在主节点创建新复制集
$ /usr/local/pgsql/bin/slonik_create_set --config /usr/local/pgsql/etc/slon_tools.conf 2 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; # TABLE ADD KEY # CREATE SET create set (id = 2, origin = 1, comment = 'Set 2 (il_longlive_schema1) for region_il'); # SET ADD TABLE echo 'Subscription set 2 (il_longlive_schema1) created'; echo 'Adding tables to the subscription set'; set add table (set id = 2, origin = 1, id = 201, full qualified name = 'il_longlive.snapshot', comment = 'Table il_longlive.snapshot with primary key'); echo 'Add primary keyed table il_longlive.snapshot'; # SET ADD SEQUENCE echo 'Adding sequences to the subscription set'; set add sequence (set id = 2, origin = 1, id = 201, full qualified name = 'il_longlive.snapshot_id_seq', comment = 'Sequence il_longlive.snapshot_id_seq'); echo 'Add sequence il_longlive.snapshot_id_seq'; echo 'All tables added'; $ /usr/local/pgsql/bin/slonik_create_set --config /usr/local/pgsql/etc/slon_tools.conf 2 | slonik <stdin>:12: Subscription set 2 (il_longlive_schema1) created <stdin>:13: Adding tables to the subscription set <stdin>:17: Add primary keyed table il_longlive.snapshot <stdin>:20: Adding sequences to the subscription set <stdin>:24: Add sequence il_longlive.snapshot_id_seq <stdin>:25: All tables added2.5. 在主节点创建新复制集的订阅者
$ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 2 2 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; subscribe set (id = 2, provider = 1, receiver = 2, forward = yes); echo 'Subscribed nodes to set 2'; $ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 2 3 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; subscribe set (id = 2, provider = 2, receiver = 3, forward = yes); echo 'Subscribed nodes to set 2'; $ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 2 2 | slonik <stdin>:7: Subscribed nodes to set 2 $ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 2 3 | slonik <stdin>:7: Subscribed nodes to set 22.6. 查看所有辅节点数据同步状态
$ psql -h 10.189.102.118 -U slonyadmin -d region_il -c "select count(*) from il_longlive.snapshot;" #主节点 Password for user slonyadmin: count ------- (1 row) $ psql -h 10.189.100.196 -U slonyadmin -d region_il -c "select count(*) from il_longlive.snapshot;" #辅节点1 ^C [postgres@ec2t-pgtest-01 bin]$ psql -h 10.189.100.195 -U slonyadmin -d region_il -c "select count(*) from il_longlive.snapshot;" Password for user slonyadmin: count ------- (1 row) $ psql -h 10.189.100.226 -U slonyadmin -d region_il -c "select count(*) from il_longlive.snapshot;" #辅节点2 Password for user slonyadmin: count ------- (1 row)2.7. 查看Slony-I系统表信息
region_il=# select * from sl_set; set_id | set_origin | set_locked | set_comment --------+------------+------------+------------------------------------------- | 1 | | Set 1 (il_longlive_schema) for region_il | 1 | | Set 2 (il_longlive_schema1) for region_il (2 rows) region_il=# select * from sl_table; tab_id | tab_reloid | tab_relname | tab_nspname | tab_set | tab_idxname | tab_altered | tab_comment --------+------------+----------------+-------------+---------+---------------------+-------------+--------------------------------------------------- | 19350 | entity | il_longlive | 1 | entity_pkey | f | Table il_longlive.entity with primary key | 19355 | entity_result | il_longlive | 1 | entity_result_pkey | f | Table il_longlive.entity_result with primary key | 19360 | result | il_longlive | 1 | result_pkey | f | Table il_longlive.result with primary key | 19365 | result_logbook | il_longlive | 1 | result_logbook_pkey | f | Table il_longlive.result_logbook with primary key | 19380 | snapshot | il_longlive | 2 | snapshot_pkey | f | Table il_longlive.snapshot with primary key (5 rows)2.8. 合并复制集
$ /usr/local/pgsql/bin/slonik_merge_sets --config /usr/local/pgsql/etc/slon_tools.conf 1 1 2 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; merge set (id = 1, add id = 2, origin = 1); echo 'Replication set 2 merged into 1 on origin 1. Set 2 no longer exists.'; $ /usr/local/pgsql/bin/slonik_merge_sets --config /usr/local/pgsql/etc/slon_tools.conf 1 1 2 | slonik <stdin>:6: Replication set 2 merged into 1 on origin 1. Set 2 no longer exists.2.9. 查看Slony-I系统表信息
region_il=# select * from sl_set; set_id | set_origin | set_locked | set_comment --------+------------+------------+------------------------------------------ | 1 | | Set 1 (il_longlive_schema) for region_il (1 row) region_il=# select * from sl_table; tab_id | tab_reloid | tab_relname | tab_nspname | tab_set | tab_idxname | tab_altered | tab_comment --------+------------+----------------+-------------+---------+---------------------+-------------+--------------------------------------------------- | 19350 | entity | il_longlive | 1 | entity_pkey | f | Table il_longlive.entity with primary key | 19355 | entity_result | il_longlive | 1 | entity_result_pkey | f | Table il_longlive.entity_result with primary key | 19360 | result | il_longlive | 1 | result_pkey | f | Table il_longlive.result with primary key | 19365 | result_logbook | il_longlive | 1 | result_logbook_pkey | f | Table il_longlive.result_logbook with primary key | 19380 | snapshot | il_longlive | 1 | snapshot_pkey | f | Table il_longlive.snapshot with primary key (5 rows)3. 向现有集群中移除一个复制表
3.1. 查看Slony-I系统视图确认要移除的表
region_il=# select * from sl_table; tab_id | tab_reloid | tab_relname | tab_nspname | tab_set | tab_idxname | tab_altered | tab_comment --------+------------+----------------+-------------+---------+---------------------+-------------+--------------------------------------------------- | 19350 | entity | il_longlive | 1 | entity_pkey | f | Table il_longlive.entity with primary key | 19355 | entity_result | il_longlive | 1 | entity_result_pkey | f | Table il_longlive.entity_result with primary key | 19360 | result | il_longlive | 1 | result_pkey | f | Table il_longlive.result with primary key | 19365 | result_logbook | il_longlive | 1 | result_logbook_pkey | f | Table il_longlive.result_logbook with primary key | 19380 | snapshot | il_longlive | 1 | snapshot_pkey | f | Table il_longlive.snapshot with primary key (5 rows)3.2. 移除集群中指定的表
$ /usr/local/pgsql/bin/slonik_drop_table --config /usr/local/pgsql/etc/slon_tools.conf 201 1 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; # DROP TABLE try { SET DROP TABLE (id = 201, origin = 1); } on error { echo 'Could not drop table 201 for region_il!'; exit 1; } $ /usr/local/pgsql/bin/slonik_drop_table --config /usr/local/pgsql/etc/slon_tools.conf 201 1 | slonik3.3. 验证集群中指定的表是否移除
region_il=# select * from sl_table; tab_id | tab_reloid | tab_relname | tab_nspname | tab_set | tab_idxname | tab_altered | tab_comment --------+------------+----------------+-------------+---------+---------------------+-------------+--------------------------------------------------- | 19350 | entity | il_longlive | 1 | entity_pkey | f | Table il_longlive.entity with primary key | 19355 | entity_result | il_longlive | 1 | entity_result_pkey | f | Table il_longlive.entity_result with primary key | 19360 | result | il_longlive | 1 | result_pkey | f | Table il_longlive.result with primary key | 19365 | result_logbook | il_longlive | 1 | result_logbook_pkey | f | Table il_longlive.result_logbook with primary key (4 rows) region_il=# \d il_longlive.snapshot #可以看到对应的表上的触发器也被移除了 Table "il_longlive.snapshot" Column | Type | Modifiers -----------------+--------------------------------+------------------------------------------------------------------- id | integer | not null default nextval('il_longlive.snapshot_id_seq'::regclass) feed_schema | character varying(255) | job | character varying(255) | table_name | character varying(255) | max_primary_key | bigint | create_time | timestamp(0) without time zone | update_time | timestamp(0) without time zone | Indexes: "snapshot_pkey" PRIMARY KEY, btree (id)4. 向现有集群中移除一个节点
4.1. 查看Slony-I系统视图确认要移除的节点
region_il=# select * from sl_node; no_id | no_active | no_comment | no_failed -------+-----------+-----------------------------------+----------- | t | Node 1 - region_il@10.189.102.118 | f | t | Node 2 - region_il@10.189.100.195 | f | t | Node 3 - region_il@10.189.100.226 | f (3 rows)4.2. 移除集群中指定的节点
$ /usr/local/pgsql/bin/slonik_drop_node --config /usr/local/pgsql/etc/slon_tools.conf 3 1 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; drop node (id = 3, event node = 1); echo 'dropped node 3 cluster'; $ /usr/local/pgsql/bin/slonik_drop_node --config /usr/local/pgsql/etc/slon_tools.conf 3 1 | slonik debug: waiting for 3,5000019408 on 1 debug: waiting for 3,5000019408 on 1 <stdin>:6: dropped node 3 cluster4.3. 验证集群中指定的节点是否移除(此时被移除节点的Slony-I系统表等信息已经被移除)
region_il=# select * from sl_node; no_id | no_active | no_comment | no_failed -------+-----------+-----------------------------------+----------- | t | Node 1 - region_il@10.189.102.118 | f | t | Node 2 - region_il@10.189.100.195 | f (2 rows)4.4. 停止被关闭节点的slon进程
$ ps -ef | grep slon postgres 6033 1 0 Jun02 "htmlcode">region_il=# select * from sl_node; no_id | no_active | no_comment | no_failed -------+-----------+-----------------------------------+----------- | t | Node 1 - region_il@10.189.102.118 | f | t | Node 2 - region_il@10.189.100.195 | f (2 rows)5.3. 向Slony-I配置文件slon_tools.conf 增加新节点信息,并将新配置文件更新到所有节点
$ cat /usr/local/pgsql/etc/slon_tools.conf ........................................... add_node(node => 3, parent => 1, host => '10.189.100.226', dbname => 'region_il', port => 5432, user => 'slonyadmin', password => '888888',); ............................................5.4. 在主节点上增加向集群新节点信息
$ /usr/local/pgsql/bin/slonik_add_node --config /usr/local/pgsql/etc/slon_tools.conf 3 1 # ADD NODE cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; try { store node (id = 3, event node = 1, comment = 'Node 3 - region_il@10.189.100.226'); } on error { echo 'Failed to add node to cluster'; exit 1; } # STORE PATHS store path (server = 3, client = 1, conninfo = 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 1, client = 3, conninfo = 'host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 3, client = 2, conninfo = 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'); store path (server = 2, client = 3, conninfo = 'host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'); echo 'added node 3 to cluster'; echo 'Please start a slon replication daemon for node 3'; $ /usr/local/pgsql/bin/slonik_add_node --config /usr/local/pgsql/etc/slon_tools.conf 3 1 | slonik <stdin>:7: Possible unsupported PostgreSQL version (90505) 9.5, defaulting to 8.4 support <stdin>:19: added node 3 to cluster <stdin>:20: Please start a slon replication daemon for node 35.5. 在新节点启动slon进程
$ slon_start 3 Invoke slon for node 3 - /usr/local/pgsql/bin/slon -p /usr/local/pgsql/data/region_il_node3.pid -s 1000 -d0 region_il 'host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888' > /usr/local/pgsql/data/pg_log/node3/region_il-2017-06-17.log 2>&1 & Slon successfully started for cluster region_il, node node3 PID [13136] Start the watchdog process as well...5.6. 在主节点添加新节点的复制订阅信息
$ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/etc/slon_tools.conf 1 3 Can't locate /usr/local/pgsql/etc/etc/slon_tools.conf in @INC (@INC contains: /usr/local/lib64/perl5 /usr/local/share/perl5 /usr/lib64/perl5/vendor_perl /usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5 .) at /usr/local/pgsql/bin/slonik_subscribe_set line 29. [postgres@ec2t-pgtest-01 bin]$ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 1 3 cluster name = region_il; node 1 admin conninfo='host=10.189.102.118 dbname=region_il user=slonyadmin port=5432 password=888888'; node 2 admin conninfo='host=10.189.100.195 dbname=region_il user=slonyadmin port=5432 password=888888'; node 3 admin conninfo='host=10.189.100.226 dbname=region_il user=slonyadmin port=5432 password=888888'; subscribe set (id = 1, provider = 1, receiver = 3, forward = yes); echo 'Subscribed nodes to set 1'; [postgres@ec2t-pgtest-01 bin]$ /usr/local/pgsql/bin/slonik_subscribe_set --config /usr/local/pgsql/etc/slon_tools.conf 1 3 | slonik <stdin>:7: Subscribed nodes to set 15.7. 验证数据同步(略)
总结
以上就是这篇文章的全部内容了，希望本文的内容对大家的学习或者工作具有一定的参考学习价值，如果有疑问大家可以留言交流，谢谢大家对的支持。
免责声明：本站文章均来自网站采集或用户投稿，网站不提供任何软件下载或自行开发的软件！
如有用户或公司发现本站内容信息存在侵权行为，请邮件告知！ 858582#qq.com
白云城资源网 Copyright www.dyhadc.com
暂无“PostgreSQL中Slony-I同步复制部署教程”评论...
一口气升级7个大模型SaaS应用，百度智能云：突出一个“开箱即用” - 2024/11/5
这一波大模型产业落地浪潮里，不少企业其实处在 “干瞪眼“的状态。
一种情况是，很多大模型产品看得见却摸不着，在台上一个个遥遥领先——今天Sora技精四座，明天英伟达的机器人又赢得满堂彩，可是到了台下一问：啥时候能用上啊？答曰：遥遥无期。
另一种情况是，企业想用上大模型，却又难免瞻前顾后——既要考虑场景融合，又得兼顾安全性，还要考虑打通现有系统，再加上各种部署成本和繁琐的采购流程……最后只能拂袖：罢了，再等等吧。
这一波大模型产业落地浪潮里，不少企业其实处在 “干瞪眼“的状态。
一种情况是，很多大模型产品看得见却摸不着，在台上一个个遥遥领先——今天Sora技精四座，明天英伟达的机器人又赢得满堂彩，可是到了台下一问：啥时候能用上啊？答曰：遥遥无期。
另一种情况是，企业想用上大模型，却又难免瞻前顾后——既要考虑场景融合，又得兼顾安全性，还要考虑打通现有系统，再加上各种部署成本和繁琐的采购流程……最后只能拂袖：罢了，再等等吧。
稳了！魔兽国服回归的3条重磅消息！官宣时间再确认！
昨天有一位朋友在大神群里分享，自己亚服账号被封号之后居然弹出了国服的封号信息对话框。
这里面让他访问的是一个国服的战网网址，com.cn和后面的zh都非常明白地表明这就是国服战网。
而他在复制这个网址并且进行登录之后，确实是网易的网址，也就是我们熟悉的停服之后国服发布的暴雪游戏产品运营到期开放退款的说明。这是一件比较奇怪的事情，因为以前都没有出现这样的情况，现在突然提示跳转到国服战网的网址，是不是说明了简体中文客户端已经开始进行更新了呢？
更新日志
2024年11月05日
2024年11月05日
- 雨林唱片《赏》新曲+精选集SACD版[ISO][2.3G]
- 罗大佑与OK男女合唱团.1995-再会吧!素兰【音乐工厂】【WAV+CUE】
- 草蜢.1993-宝贝对不起（国）【宝丽金】【WAV+CUE】
- 杨培安.2009-抒·情（EP）【擎天娱乐】【WAV+CUE】
- 周慧敏《EndlessDream》[WAV+CUE]
- 彭芳《纯色角3》2007[WAV+CUE]
- 江志丰2008-今生为你[豪记][WAV+CUE]
- 罗大佑1994《恋曲2000》音乐工厂[WAV+CUE][1G]
- 群星《一首歌一个故事》赵英俊某些作品重唱企划[FLAC分轨][1G]
- 群星《网易云英文歌曲播放量TOP100》[MP3][1G]
- 方大同.2024-梦想家TheDreamer【赋音乐】【FLAC分轨】
- 李慧珍.2007-爱死了【华谊兄弟】【WAV+CUE】
- 王大文.2019-国际太空站【环球】【FLAC分轨】
- 群星《2022超好听的十倍音质网络歌曲(163)》U盘音乐[WAV分轨][1.1G]
- 童丽《啼笑姻缘》头版限量编号24K金碟[低速原抓WAV+CUE][1.1G]