Today I needed to find duplicates files from:
– list of URLs accessed
– list of category URLs I needed to refresh data for
here's my code snippet:
(This assumes LOG.txt is the file with URLs logged and CATEGORIES.txt is the list of all URLs that would pottentially need to be refreshed)
for line in `cat CATEGORIES.txt`; do grep $line LOG.txt; done
You can see it in action and output to a file by adding:
| tee OUTPUT.txt
First install autoconf, php pear and curl
pacman -Syu autoconf php-pear curl
Then install pecl_http and iconv
pear install pecl.php.net/pecl_iconv
pear install pecl.php.net/pecl_http, iconv
Edit your php.ini
nano /etc/php/php.ini
Add
extension=http.so
Restart apache
rc.d restart httpd
Install Imagick
# pacman -S imagemagick libpng librsvg libwmf ghostscript libxml2 jasper openexr
#sudo pear install pecl.php.net/imagick
I tried using the same changes for my linux boxes on my macbook air and they didn't work, so I googled around and stumbled upon using this instead:
c_cyan=`tput setaf 6`
c_red=`tput setaf 1`
c_green=`tput setaf 2`
c_sgr0=`tput sgr0`
parse_git_branch (){
if git rev-parse --git-dir >/dev/null 2>&1
then
gitver=$(git branch 2>/dev/null| sed -n '/^\*/s/^\* //p')
else
return 0
fi
echo -e $gitver
}
branch_color (){
if git rev-parse --git-dir >/dev/null 2>&1
then
color=""
if git diff --quiet 2>/dev/null >&2
then
color="${c_green}"
else
color=${c_red}
fi
else
return 0
fi
echo -ne $color
}
#Branch First
#PS1='[\[$(branch_color)\]$(parse_git_branch)\[${c_sgr0}\]] \u@\[${c_red}\]\w\[${c_sgr0}\]: '
#Branch Last
PS1=' \u@\[${c_red}\]\w\[${c_sgr0}\]: [\[$(branch_color)\]$(parse_git_branch)\[${c_sgr0}\]] '
original: http://asemanfar.com/Current-Git-Branch-in-Bash-Prompt
Follow these steps to setup a decent backup solution for your MT hosting (you can adapt this to any other hosting that offers SSH and RSYNC access).
First you'll want your linux box to be able to SSH into your hosting without a password using a SSH key instead.
Then, just clone the following repository somewhere in your linux box
https://github.com/PortNumber53/mt-utils
Customize the file:
mt.sh.sample
Next, make sure the variables in
0-variables.sh
match your enviroment.
Then you just need to run:
mt.sh.sample
For extra peace of mind, set it to run from a cron job.
After cloning the repository, let's add the Kohana core, a few modules and customize the installation:
1.
git submodule add git://github.com/kohana/core.git modules/system
git submodule add git://github.com/kohana/database.git modules/database
git submodule add git://github.com/kohana/cache.git modules/cache
git submodule add git://github.com/kohana/image.git modules/image
git submodule add git://github.com/kohana/pagination.git modules/pagination
2. Initialize the modules
$ git submodule init
Warning: require(/bootstrap.php): failed to open stream: No such file or directory in /home/httpd/domains/menospior.dev/html/index.php on line 102
Fatal error: require(): Failed opening required '/bootstrap.php' (include_path='.:/php/library/php-redis:/usr/share/pear') in/home/httpd/domains/menospior.dev/html/index.php on line 102
Chage a few settings:
html/index.php
$application = '../modules/application';
$modules = '../modules';
$system = '../modues/system';
Customize some kohana settings:
../modules/application/bootstrap.php
date_default_timezone_set('America/Los_Angeles');
Now create the logs folder
$ mkdir logs
At this point you should have a 404 error as there's no controller:
SYSPATH/classes/kohana/request/client/internal.php [ 87 ]
82
83 try
84 {
85 if ( ! class_exists($prefix.$controller))
86 {
87 throw new HTTP_Exception_404('The requested URL :uri was not found on this server.',
88 array(':uri' => $request->uri()));
89 }
90
91 // Load the controller using reflection
92 $class = new ReflectionClass($prefix.$controller);
SYSPATH/classes/kohana/request/client.php [ 64 ] » Kohana_Request_Client_Internal->execute_request(arguments)
SYSPATH/classes/kohana/request.php [ 1138 ] » Kohana_Request_Client->execute(arguments)
DOCROOT/index.php [ 109 ] » Kohana_Request->execute()
This is optional:
Problems you may want to fix:
The iconv extension is not loaded.
Uncomment the line
extension=iconv.so
in your php.ini file and restart Apache
Kohana requires mcrypt for the Encrypt class.
Uncomment the line
extension=mcrypt.so
in your php.ini file and restart Apache
Kohana requires GD v2 for the Image class.
Uncomment the line
extension=gd.so
in your php.ini file and restart Apache
Kohana can use the http extension for the Request_Client_External class.
Install the pecl_http extension
/usr/lib/php/modules/http.so
You should add "extension=http.so" to php.ini
Fix install.php folder checks:
This is a version of this post http://blog.portnumber53.com/2012/01/15/intranet-setup-a-dnscache-for-your-network-using-djbdns/ for the ArchLinuxARM:
Essentials:
# pacman -Syu fakeroot patch make gcc dnsutils
Daemontools installation
$ wget https://aur.archlinux.org/packages/da/daemontools/daemontools.tar.gz
$ tar -xvzf daemontools.tar.gz
$ cd daemontools
$ nano PKGBUILD
——————–
arch=('arm')
——————–
$ makepkg
$ sudo pacman -U daemontools-0.76-5-arm.pkg.tar.xz
You shoould see this:
loading packages…
resolving dependencies…
looking for inter-conflicts…
Targets (1): daemontools-0.76-5
Total Installed Size: 0.30 MiB
Proceed with installation? [Y/n] y
(1/1) checking package integrity [######################################] 100%
(1/1) loading package files [######################################] 100%
(1/1) checking for file conflicts [######################################] 100%
(1/1) installing daemontools [######################################] 100%
Adding svscanboot to inittab…
init should start svscan now.
[root@heart daemontools]#
ucspi-tcp 0.88-3
UCSPI-TCP
$ wget https://aur.archlinux.org/packages/uc/ucspi-tcp/ucspi-tcp.tar.gz
$ tar -xvzf ucspi-tcp.tar.gz
$ cd ucspi-tcp
$ nano PKGBUILD
——————–
arch=('arm')
——————–
change line 36 to:
install -m 755 -D $bin $pkgdir/usr/bin/$bin
(refer to the comments at https://aur.archlinux.org/packages.php?ID=8330 )
$ sudo pacman -U ucspi-tcp-0.88-3-arm.pkg.tar.xz
Now for the real thing:
$ wget https://aur.archlinux.org/packages/dj/djbdns-ipv4/djbdns-ipv4.tar.gz
$ tar -xvzf djbdns-ipv4.tar.gz
$ cd djbdns-ipv4/
I built mine on my PogoPlug, so I needed to edit PKGBUILD and edited: arch=('arm')
$ makepkg
$ sudo pacman -U djbdns-ipv4-1.05-2-arm.pkg.tar.xz
============
>>> Configure and run tinydns:
tinydns-conf _tinydns _dnslog /service/tinydns [IP]
ln -s /service/tinydns /etc/
>>> Configure and run dnscache as a recursive cache:
dnscache-conf _dnscache _dnslog /service/dnscache [IP]
ln -s /service/dnscache /etc/
>>> To change from recursive to forward-only:
echo 1 > /service/dnscache/env/FORWARDONLY
echo IP_address_of_a_recursive_DNS server > /service/dnscache/root/servers/@
echo IP_address_of_another_recursive_DNS server >> /service/dnscache/root/servers/@
>>> To add a local authoritative server to a recursive or forward-only setup:
echo IP_address_of_a_local_authoritative_server > /service/dnscache/root/servers/name.of.toplevel.domain
If [IP] is omitted dnscache will run on localhost.
More help can be found at http://cr.yp.to/djbdns.html
Read /usr/share/djbdns/README for more information about this package.
=================
Let's set up the dns cache on the box's IP (10.0.0.240 in the example)
# dnscache-conf _dnscache _dnslog /etc/dnscache 10.0.0.240
Start it
# ln -s /etc/dnscache /service/
ps ax should show something like:
9438 ? S 0:00 supervise dnscache
9439 ? S 0:00 supervise log
9440 ? S 0:00 multilog t ./main
9441 ? S 0:00 /usr/bin/dnscache
Tell DNSCACHE to accept queries from your intranet:
touch /etc/dnscache/root/ip/10
Let's tail its log:
# tail -f /service/dnscache/log/main/current
And test it, opening another terminal
# dnsip www.cnn.com
As of this writing you should see:
157.166.255.18 157.166.255.19 157.166.226.26 157.166.226.25
and a bunch of log output
Follow these steps to manage your kohana application using GIT:
1. Create an empty directory
$ mkdir insertcoin.local && cd insertcoin.local
2. run git init to create the bare structure for a git repository
$ git init
3. create a folder for the core
$ mkdir vendor
4. Add submodules:
$ cd insertcoin.local/
$ git submodule add git://github.com/kohana/core.git vendor/kohana
$ git submodule add git://github.com/kohana/database.git modules/database
$ git submodule add git://github.com/kohana/cache.git modules/cache
$ git submodule add git://github.com/kohana/image.git modules/image
$ git submodule add git://github.com/kohana/pagination.git modules/pagination
5. Init these submodules
$ git submodule init
$ git commit -m 'Added core kohana and initial submodules [cache,database,image,pagination]'
6. push these changes
$ git pull origin master
$ git push origin master
7 Get some more files:
$ mkdir html
$ cd html
$ wget https://raw.github.com/kohana/kohana/3.2/master/index.php
$ cd ../modules/application/
$ wget https://raw.github.com/kohana/kohana/3.2/master/application/bootstrap.php
Chage a few settings:
html/index.php
$application = '../modules/application';
$modules = '../modules';
$system = '../vendor/kohana';
modules/application/bootstrap.php
date_default_timezone_set('America/Los_Angeles');
$ git clone git://github.com/kohana/core.git vendor/kohana/
$ mkdir -p modules/application/classes/controller
$ mkdir -p modules/application/config
$ touch -p modules/application/classes/controller/.gitignore
$ mkdir -p modles/application/views/templates/default
$ touch -p modules/applcation/views/template/default/.gitignore
wget https://github.com/kohana/kohana/raw/3.2/master/index.php
git clone mt:~/domains/git.portnumber53.com/html/in.sert.co.in insertcoin.local
$ cd modules/application
$ wget https://raw.github.com/kohana/kohana/3.2/master/application/bootstrap.php
Setting the push script for the site:
$ mkdir export
$ git clone http://git.portnumber53.com/in.sert.co.in.git export/in.sert.co.in
$ git clone http://git.portnumber53.com/in.sert.co.in.git ~/domains/in.sert.co.in.temp
I have improved my Kohana 3.2.x installation steps.
This new way takes advantages of submodules with Git thus making your life easier when upgrading to a new version, assuming it's backwards compatible.
This guide will assume you have the following structure for your website:
root/html <!– public html folder
root/modules <!– kohana files
root/source <!– environment dependable files
Ready? here we go:
1. Create an empty git repository following this:
2. Create a submodule with Kohana's core using:
$ git submodule add git://github.com/kohana/core.git modules/system
3. Add all the modules you'll need:
$ git submodule add git://github.com/kohana/database.git modules/database
$ git submodule add git://github.com/kohana/image.git modules/image
$ git submodule add git://github.com/kohana/pagination.git modules/pagination
4. Initialize the submodules
$ git submodule initSubmodule 'modules/database' (git://github.com/kohana/database.git) registered for path 'modules/database'
Submodule 'modules/image' (git://github.com/kohana/image.git) registered for path 'modules/image'
Submodule 'modules/pagination' (git://github.com/kohana/pagination.git) registered for path 'modules/pagination'
Submodule 'modules/system' (git://github.com/kohana/core.git) registered for path 'modules/system'
5. Commit these changes
$ git commit -m 'Added initial submodules'
7. Now we need index.php and bootstrap.php files:
$ mkdir html
$ wget http://github.com/kohana/kohana/raw/3.0/master/index.php -O html/index.php
$ wget http://github.com/kohana/kohana/raw/3.0/master/application/bootstrap.php -O modules/application/bootstrap.php
8. Fix file paths on html/index.php
<?php
$application = '../modules/application';
$modules = '../modules';
$system = '../modules/system';
9. Change the default timezone
<?php
date_default_timezone_set('America/Los_Angeles');
10. Add a default .htaccess
wget https://raw.github.com/kohana/kohana/3.2/master/example.htaccess -O html/.htaccess