Unix Job processing
I learned something new from my brother the other day about unix job processing. We had written a script to do some specific analysis of the logs for dealazon.com, and it would take about 30 seconds to a minute for the script to complete.
I had known for quite some time that you can run a script in the background by appending an & to the end of your command, for instance:
$> ./some_slow_script.pl &
I often run scripts that take a while, but forget to append the & so they run in the background. Well it turns out you can hit Ctrl+Z this will pause the process that is currently running, and give you a shell. Now you can type bg to send that process into the background. Now you can use the jobs command to see how its doing, or the fg command to send it back into the foreground.
So for example you might do something like this:
$> ./some_slow_script.pl hits Ctrl+Z [1]+ Stopped ./some_slow_script.pl $> $> bg [1]+ ./some_slow_script.pl & $> jobs [1]+ Running ./some_slow_script.pl & $> jobs [1]+ Done ./some_slow_script.pl
add to del.icio.usRelated Entries
- The 15 Essential UNIX commands - July 29, 2005
- Shell Script for backpack todo lists - August 20, 2005
Trackbacks
Trackback Address: 502/B96D1B5C28FD0AEA9905A36741111DFF
Comments
On 11/29/2005 at 12:20:36 AM EST jehiah wrote:
1
You can also background multiple processes, and recover them individually
hits Ctrl+Z [1]+ Stopped ./some_slow_script.pl hits Ctrl+Z on another process [2]+ Stopped ./another_slow_script.pl
Now you can recover them individually $ fg 1 or $ fg 2
On 11/29/2005 at 4:29:17 AM EST Brandon Harper wrote:
2
You should also check into screen if you'd like to play with background processes some more and so forth:
http://linuxreviews.org/man/screen/
Basically you can use it to start processes in different virtual terminals and detach from them to new terminals or just logout altogether. I used to use screen way back when to run IRC bots... I could start one in a screen, detach from it, and logout of my shell and it would still be running.
Actually, IIRC, you can also start background processes which will still run after you logout by prepending a 'nohup' in front of them. ie: 'nohup ./script &'. However screen gives you the ability to reattach to the process if you need to interact with it, check it's progress, etc.
On 11/29/2005 at 9:40:03 AM EST Pete Freitag wrote:
3
Cool tips Jehiah and Brandon - Thanks!
On 11/29/2005 at 10:58:21 AM EST Yves wrote:
4
Nice...
Stuff like this can certainly come in handy.
On 03/29/2006 at 1:26:22 AM EST kishore wrote:
5
how do i redirect the "[1] xxxx" output to a file (command &) is not the solution, as I NEED the id in a file
On 02/24/2009 at 4:08:29 PM EST Anonymous wrote:
6
great advice on background running of a script.
Post a Comment
Recent Entries
- Cache Template in Request Setting Explained
- What Version of Java is ColdFusion Using?
- ColdFusion 9 Performance Brief from Adobe
- Request Filtering in IIS 7 Howto
- J2EE Session Cookies on ColdFusion / JRun
- Hands on ColdFusion Security Training
- ColdFusion 9 Solr Vulnerability - Are you at Risk?
- FCKEditor Year 2010 Bug for Firefox 3.6 with ColdFusion
hits Ctrl+Z [1]+ Stopped ./some_slow_script.pl hits Ctrl+Z on another process [2]+ Stopped ./another_slow_script.pl
Now you can recover them individually $ fg 1 or $ fg 2
http://linuxreviews.org/man/screen/
Basically you can use it to start processes in different virtual terminals and detach from them to new terminals or just logout altogether. I used to use screen way back when to run IRC bots... I could start one in a screen, detach from it, and logout of my shell and it would still be running.
Actually, IIRC, you can also start background processes which will still run after you logout by prepending a 'nohup' in front of them. ie: 'nohup ./script &'. However screen gives you the ability to reattach to the process if you need to interact with it, check it's progress, etc.
Stuff like this can certainly come in handy.
