请教一个初级问题，并发插入能提高插入速度吗？

[笑天居士]

Innodb引擎，开启log_bin

比如，一个连接，插入5000条，用50S

那如果我多线程并发，如10个线程，10个连接，每个连接同时并发的插入，每个连接插入500条，那会提高10倍速度吗？或者有明显的提高呢？



我的测试结果是，1个，10个，20个，50个，100个线程，插入总时间都是一样，感觉就是和单线程一样，是顺序着插入的

这是为什么呢？根本就没有并发作用啊？其它数据库如oracle也是这样吗，是我的配置不对吗？怎样优化呢？


那网站怎么实现大并发啊？


配置文件如下：


# Example MySQL config file for medium systems.
#
# This is for a system with little memory (32M - 64M) where MySQL plays
# an important part, or systems up to 128M where MySQL is used together with
# other programs (such as a web server)
#
# You can copy this file to
# /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options (in this
# installation this directory is /var/lib/mysql) or
# ~/.my.cnf to set user-specific options.
#
# In this file, you can use all long options that a program supports.
# If you want to know which options a program supports, run the program
# with the "--help" option.

# The following options will be passed to all MySQL clients
[client]
user = mysql
password = mysql
port = 3306
socket = /tmp/mysql.sock

# Here follows entries for some specific programs

# The MySQL server
[mysqld]
port = 3306
socket = /tmp/mysql.sock
skip-locking
key_buffer = 1000M
max_allowed_packet = 1M
table_cache = 64
sort_buffer_size = 512K
net_buffer_length = 8K
read_buffer_size = 256K
read_rnd_buffer_size = 512K
myisam_sort_buffer_size = 8M
default-storage-engine = InnoDB
innodb_file_per_table
max_connections = 1000

# Don't listen on a TCP/IP port at all. This can be a security enhancement,
# if all processes that need to connect to mysqld run on the same host.
# All interaction with mysqld must be made via Unix sockets or named pipes.
# Note that using this option without enabling named pipes on Windows
# (via the "enable-named-pipe" option) will render mysqld useless!
#
#skip-networking

# Replication Master Server (default)
# binary logging is required for replication
# log-bin=mysql-bin
# sync_binlog = 1

# required unique id between 1 and 2^32 - 1
# defaults to 1 if master-host is not set
# but will not function as a master if omitted
# server-id = 1

# Replication Slave (comment out master section to use this)
#
# To configure this host as a replication slave, you can choose between
# two methods :
#
# 1) Use the CHANGE MASTER TO command (fully described in our manual) -
# the syntax is:
#
# CHANGE MASTER TO MASTER_HOST=<host>, MASTER_PORT=<port>,
# MASTER_USER=<user>, MASTER_PASSWORD=<password> ;
#
# where you replace <host>, <user>, <password> by quoted strings and
# <port> by the master's port number (3306 by default).
#
# Example:
#
# CHANGE MASTER TO MASTER_HOST='125.564.12.1', MASTER_PORT=3306,
# MASTER_USER='joe', MASTER_PASSWORD='secret';
#
# OR
#
# 2) Set the variables below. However, in case you choose this method, then
# start replication for the first time (even unsuccessfully, for example
# if you mistyped the password in master-password and the slave fails to
# connect), the slave will create a master.info file, and any later
# change in this file to the variables' values below will be ignored and
# overridden by the content of the master.info file, unless you shutdown
# the slave server, delete master.info and restart the slaver server.
# For that reason, you may want to leave the lines below untouched
# (commented) and instead use CHANGE MASTER TO (see above)
#
# required unique id between 2 and 2^32 - 1
# (and different from the master)
# defaults to 2 if master-host is set
# but will not function as a slave if omitted
#server-id = 2
#
# The replication master for this slave - required
#master-host = lrfsim
#
# The username the slave will use for authentication when connecting
# to the master - required
#master-user = mysql
#
# The password the slave will authenticate with when connecting to
# the master - required
#master-password = mysql
#
# The port the master is listening on.
# optional - defaults to 3306
#master-port = 3306
#
#master-connect-retry=60
#
#replicate-do-db=jfytest
#replicate-do-db=door
#
# binary logging - not required for slaves, but recommended

# Point the following paths to different dedicated disks
#tmpdir = /tmp/
#log-update = /path-to-dedicated-directory/hostname

# Uncomment the following if you are using BDB tables
#bdb_cache_size = 4M
#bdb_max_lock = 10000

# Uncomment the following if you are using InnoDB tables
innodb_data_home_dir = /usr/local/mysql/data/
innodb_data_file_path = ibdata1:1000M:autoextend
innodb_log_group_home_dir = /usr/local/mysql/data/
innodb_log_arch_dir = /usr/local/mysql/data/
# You can set .._buffer_pool_size up to 50 - 80 %
# of RAM but beware of setting memory usage too high
#innodb_buffer_pool_size = 2000M
#innodb_buffer_pool_size = 1000M
#innodb_additional_mem_pool_size = 80M
# Set .._log_file_size to 25 % of buffer pool size
innodb_log_file_size = 500M
innodb_log_buffer_size = 8M
#innodb_flush_log_at_trx_commit = 2
#innodb_lock_wait_timeout = 50
#innodb_flush_method=O_DSYNC

[mysqldump]
quick
max_allowed_packet = 16M

[mysql]
no-auto-rehash
# Remove the next comment character if you are not familiar with SQL
#safe-updates

[isamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
wrGite_buffer = 2M

[myisamchk]
key_buffer = 20M
sort_buffer_size = 20M
read_buffer = 2M
write_buffer = 2M

[mysqlhotcopy]
interactive-timeout

[笑天居士]

如果我关掉log_bin，多线程情况下，速度会有成倍提升

[ACMAIN_CHM]

不管你有多少个并发进程，但MYSQL针对同一个表，总是顺序进行的，插入完一条才能再插入第二条。

[笑天居士]

[Quote=引用 2 楼 的回复:]
不管你有多少个并发进程，但MYSQL针对同一个表，总是顺序进行的，插入完一条才能再插入第二条。
[/Quote]

那为何我关掉logbin，多进程情况下确实比单进程插入快很多啊

[rucypli]

[Quote=引用 3 楼 的回复:]

引用 2 楼 的回复:
不管你有多少个并发进程，但MYSQL针对同一个表，总是顺序进行的，插入完一条才能再插入第二条。


那为何我关掉logbin，多进程情况下确实比单进程插入快很多啊
[/Quote]
关掉binlog 单进程也比原来的快

[笑天居士]

[Quote=引用 4 楼 的回复:]
引用 3 楼 的回复:

引用 2 楼 的回复:
不管你有多少个并发进程，但MYSQL针对同一个表，总是顺序进行的，插入完一条才能再插入第二条。


那为何我关掉logbin，多进程情况下确实比单进程插入快很多啊

关掉binlog 单进程也比原来的快
[/Quote]


但是多进程时，是成倍的快啊

[nicenight]

我认为这个现象可能是磁盘IO影响的，因为要写数据文件又要写日志文件，磁盘IO至少是不开日志的两倍以上。

另外插入的速度还与使用的方式有关，官网有说明：
load data infile 方式 > 单条语句插入多行数据 > 单条语句插入单条数据

[笑天居士]

[Quote=引用 6 楼 的回复:]
我认为这个现象可能是磁盘IO影响的，因为要写数据文件又要写日志文件，磁盘IO至少是不开日志的两倍以上。

另外插入的速度还与使用的方式有关，官网有说明：
load data infile 方式 > 单条语句插入多行数据 > 单条语句插入单条数据
[/Quote]

谢谢楼上几位

[笑天居士]

现在的问题基本弄清楚了

有两个参数
sync_binlog
innodb_flush_log_at_trx_commit


innodb_flush_log_at_trx_commit=0，是很快，并发也会起作用

但我们做的是OLTP运营系统的，要充分保证数据的完整性与一致性
innodb_flush_log_at_trx_commit=0，2，服务器掉电或mysql死掉，1S的事务数据就没了

sync_binlog=0是最快的，但是我们也要保证主从的高一致性，因此这项也不能是0或2


因此，就只能是慢点，安全点了