新人写shell脚本遇到的问题

weixin_42177498 2018-09-16 12:27:19
最近刚试着写shell脚本,尝试用shell写并发任务时遇到很多问题,请大神看一下

1.在并发后用wait不起作用。
脚本是先读取文件顺序执行,再并发执行,等并发完成后,再并发一次;但是在第一个并发后面加wait命令不起作用,还是不等第一个并发执行完成就执行第二个并发。
脚本如下:
#!/bin/bash
tyfo_transaction_order="/var/lib/hive/zjq/shell/tyfo_impala/tyfo_transaction_order.txt"
tyfo_transaction_meanwhile1="/var/lib/hive/zjq/shell/tyfo_impala/tyfo_transaction_meanwhile1.txt"
tyfo_transaction_meanwhile2="/var/lib/hive/zjq/shell/tyfo_impala/tyfo_transaction_meanwhile2.txt"

echo $(date +%H:%M:%S)
cat $tyfo_transaction_order | while read line
do
$line >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_transaction_order.log 2>&1
echo $line
done

echo $(date +%H:%M:%S)
cat $tyfo_transaction_meanwhile1 | while read line
do
$line >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_transaction_meanwhile1.log 2>&1 &
echo $line
done

wait

echo $(date +%H:%M:%S)

cat $tyfo_transaction_meanwhile2 | while read line
do
$line >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_transaction_meanwhile2.log 2>&1 &
echo $line
sleep 5
done

2.由于上面每个顺序执行和并发执行的循环里运行了很多不同脚本,就想着每个脚本存一个不同的日志文件。就遇到了下面一个问题:
传参就去的命令报错,但是单独直接运行可以执行;代码如下:
#!/bin/bash
tyfo_transaction_order="/var/lib/hive/zjq/shell/test_impala/tyfo_transaction_order.txt"
tyfo_transaction_meanwhile1="/var/lib/hive/zjq/shell/test_impala/tyfo_transaction_meanwhile1.txt"
tyfo_transaction_meanwhile2="/var/lib/hive/zjq/shell/test_impala/tyfo_transaction_meanwhile2.txt"

OLD_IFS="$IFS"
IFS=","
echo $(date +%H:%M:%S)
cat $tyfo_transaction_meanwhile1 | while read line
do
{
arr=($line)
sh_name=${arr[0]}" >"${arr[1]}" 2>&1"
echo $sh_name
$sh_name
sleep 5
} &
done
wait
echo $(date +%H:%M:%S)

cat $tyfo_transaction_meanwhile2 | while read line
do
{
arr=($line)
sh_name=${arr[0]}" >"${arr[1]}" 2>&1"
echo $sh_name
$sh_name
sleep 5
} &
done
报错:
test.sh: line 18: impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/transaction_flow_shop/tyfo_shop_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_shop_transaction.log 2>&1: No such file or directory
test.sh: line 18: impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/transaction_flow_terminal/tyfo_terminal_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_terminal_transaction.log 2>&1: No such file or directory
test.sh: line 18: > 2>&1: command not found
23:57:34
impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/check_data/etl_check_number_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/etl_check_number_transaction.log 2>&1
impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/check_data/etl_check_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/etl_check_transaction.log 2>&1
test.sh: line 33: impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/check_data/etl_check_number_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/etl_check_number_transaction.log 2>&1: No such file or directory


直接执行:impala-shell -i hadoop03 -f /usr/test/impala/sqlscript/transaction_flow_shop/tyfo_shop_transaction.sql >/var/lib/hive/zjq/shell/tyfo_impala_log/tyfo_shop_transaction.log 2>&1 是没有问题的


新人还望大神们多帮助
...全文
104 回复 打赏 收藏 转发到动态 举报
写回复
用AI写文章
回复
切换为时间正序
请发表友善的回复…
发表回复

488

社区成员

发帖
与我相关
我的任务
社区描述
硬件使用 非技术区
社区管理员
  • 非技术区社区
加入社区
  • 近7日
  • 近30日
  • 至今
社区公告
暂无公告

试试用AI创作助手写篇文章吧