William Jiang

JavaScript,PHP,Node,Perl,LAMP Web Developer – http://williamjxj.com; https://github.com/williamjxj?tab=repositories

Category Archives: System

web server .htaccess

Web Server .htaccess routing example

For a PHP micro-framework and quick solution, using web-server’s (like Apache) URL rewrite(mod_rewrite) is a good choice. Just add/edit .htaccess file in specific/various dirs to control the URL routines. A standard .htaccess file:

RewriteEngine On
RewriteBase /name/

RewriteCond %{DOCUMENT_ROOT}/name/$1.php -f
RewriteRule ^([^/]+)/([^/]+)/?$ $.php1?action=$2 [L,NC,QSA]

RewriteCond %{DOCUMENT_ROOT}/name/$1.php -f    
RewriteRule ^([^/]+)/([^/]+)/([^/]+)/?$ $1.php?action=$2&id=$3 [L,NC,QSA]

RewriteCond %{REQUEST_FILENAME} !-d 
RewriteCond %{REQUEST_FILENAME}.php -f 
RewriteRule ^(.*)$ $1.php [L]

This will convert a php HTTP request to a better meaningful URL:

//From HTTP Request:
routing.php
routing.php?action=create
routing.php?action=list&id=1

// To:
/routing
/routing/create
/routing/list/1

In this way, PHP doesn’t involve into routing request. Compared to PHP MVC framework and routine, it is faster, but less extensible and flexibility. A good URL rewriting example from CakePHP is at:

http://book.cakephp.org/2.0/en/installation/url-rewriting.html

Advertisements

SetEnvIf: Apache httpd log filter

SetEnvIf: Apache httpd log filter

My dedicated server’s (in godday.com) log files (error.log, access.log) are frequently accessed by google Googlebot and some engine spiders or from certain IPs, thus generate duplicated, useless log messages.
I have to filter these kind of messaghe to make logfile’s size increase not so quickly. I use ‘SetEnvIf‘ in httpd.conf to customize the log output to prevent such logging requests for both error.log and access.log.
Here is a detailed introduction of ‘SetEnvIf‘: How To Tell Apache To Not Log Certain Requests In Its Access Log?

The following are some ‘SetEnvIf‘ examples to filter certain IPs:

– To prevent all requests made with a certain browser, e.g. Internet Explorer, from getting logged:
SetEnvIf User_Agent “(MSIE)” dontlog

– To not log requests from any client whose hostname ends in bla.example.com, use:
SetEnvIf Remote_Host “bla.example.com$” dontlog

– To not log requests from any client whose hostname begins with example, use:
SetEnvIf Remote_Host “^example” dontlog

To not log requests from a certain IP address, use something like:
SetEnvIf Remote_Addr “192.168.0.154” dontlog

– If you don’t want requests of your robots.txt to get logged, use:
SetEnvIf Request_URI “^/robots.txt$” dontlog

– Apart from SetEnvIf, which is case-sensitive, you can use SetEnvIfNoCase which is case-insensitive.

For example, in order not to log certain search engine spiders, you could use:

SetEnvIFNoCase User-Agent "Slurp/cat" dontlog
SetEnvIFNoCase User-Agent "Ask Jeeves/Teoma" dontlog
SetEnvIFNoCase User-Agent "Googlebot" dontlog
SetEnvIFNoCase Remote_Host "fastsearch.net$" dontlog
– Or to not log certain file extensions, use something like this:
SetEnvIfNoCase Request_URI “.(gif)|(jpg)|(png)|(css)|(js)|(ico)|(eot)$” dontlog

– To not log certain referrals (e.g. from your own domain), use something like:
SetEnvIfNoCase Referer “www.mydomain.com” dontlog

Some coding tips

Some coding tips

Here I list some tips from my coding for quick retrieving:

//Useful in a virtualbox env:
# setsebool httpd_can_network_connect 1
# setsebool httpd_can_network_connect_db 1

//remove from git remote repository:
git rm -r --cached vendor
git rm --cached .gitingore

// document_root/.htaccess file as routers
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^(.*)$ index.php [QSA,L]
RewriteRule ^(.*)$ index.php/$1 [L]

//Leverage Browser Caching
<IfModule mod_expires.c>
# Enable Expirations
ExpiresActive On
# Default Expiration Time
ExpiresDefault "access plus 1 month"
# Expiration for CSS
ExpiresByType text/css "access plus 1 month”
# Expiration for JavaScript
ExpiresByType application/javascript "access plus 1 month"
</IfModule>

// steps required after installing redis-server and mongod
$ sudo groupadd mongod redis
$ sudo useradd -s /sbin/nologin -g mongod -d /var/lib/mongo
      -M -r -c "MongoD daemon" mongod
$ sudo adduser -s /sbin/nologin -g redis -d /var/lib/redis
      -M -r -c "Redis Server" redis

//a typical JS closure: nextId()
var nextId=(function() {
  var counter = 1;
  return function() {
    return counter ++;
  };
}());

// ajax should write like this:
$.ajax({
  type:'POST', url:'',
  data: JSON.stringify(post_data),
  contentType: "application/json; charset=utf-8",
  dataType: "json"
}) //promise
.done(function(data) {}) //then
.done(function(data) {}) //...
.fail(function (jqxhr, textStatus, errorThrown) {})
.always(function (data) {});

// dynamic loading js:
var url="http://dev.jquery.com/view/trunk/plugins/color/jquery.color.js";
$.getScript(url, function() {
  $("#go").click(function(){
    $(".block").animate( { backgroundColor: "pink" }, 1000)
      .delay(500)
      .animate( { backgroundColor: "blue" }, 1000);
  });
});

A powerful .exrc file for vim

a powerful .exrc file for vim

The following .exrc file make my CentOS web development environment is powerful and pretty quick. A lot of shortcut ways to write codes. Pretty cool: vi ~/.exrc to see –>

set autoindent
set noautowrite
set shiftwidth=4
set tabstop=4
set wrapmargin=2

abbr _href <a href=""></a>
abbr _img <img src="" border="0" width="" height="" alt="[image]" align="top">

ab get_jquery <script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"></script>
ab get_bcss <link href="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/css/bootstrap-combined.min.css" rel="stylesheet">
ab get_bjs  <script src="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/js/bootstrap.min.js"></script>

map ,h5  o<!DOCTYPE html>
<head>
  <meta charset="utf-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge,chrome=1">
<title></title>
<meta name="description" content="">
<meta name="viewport" content="width=device-width">
<script src="//ajax.googleapis.com/ajax/libs/jquery/2.0.0/jquery.min.js"></script>
<link href="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/css/bootstrap-combined.min.css" rel="stylesheet">
<script src="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/js/bootstrap.min.js"></script>
 </head>

map ,b5 o  <body>
  <div class="container">
    <div class="row">
      <div class="span4"></div>
</div>
</div>
</body>
<script>jQuery(function($){});</script>
</html>ESC

abbr, ab, and map are truely shortcut setting.

JavaScript: Empty Object Comparison

JavaScript: Empty Object Comparison

The following is my summary of empty object comparsion. They used in node command line.
They are slightly difference. For prototype inheritance, always use var o={};

var o; //undefined var o=null; //for empty object var o={}; //distinct a object
var o; var o=null; var o = {};
o.__proto__; //Error o.__proto__; //Error o.__proto__; // {}
o.constructor; //Error o.constructor; //Error o.constructor; // [Function: Object]
o.instanceof Object; //false o.instanceof Object; //false; o instanceof Object; // true
typeof o; //’undefined’ typeof t; //’object’ typeof o; // ‘object’
o; //undefined o; //null o; //{}
o.length; // Error o.length; //Error o.length; //undefined
o.toString(); //Error o.toString(); //Error o.toString(); //[object Object]

How to determinate if a given linux is 32 or 64 bits?

How to determinate if a given linux is 32 or 64 bits?

When I use install Oracle Virtualbox (CentOS, Ubuntu) in windows, sometimes I am confused the installed linux is
32- or 64-bits when installed relative package such as various rpm, mongodb source etc. the command ‘uname -a‘ display:

  $ uname -a
Linux hostname 2.6.32-358.14.1.el6.i686 #1 SMP 
  ... i686 i686 i386 GNU/Linux 

Make no sense to me. In stackoverflow.com I found the answer:

  $ uname -m
  x86_64 ==> 64-bit kernel
  i686   ==> 32-bit kernel

so if return i686, it means 32-bit, while X86_64 means 64-bits. This saves me a lot when install rpm packages.
By the way, the command ‘lscpu‘ also works.

redis server predis permission denied: setsebool

redis server predis permission denied: setsebool

It is quite strange to throw errors when php codes access local MongoDB or Redis-server. e.g.: locally in CentOS 6.3, the redis-server runs as daemon, when I use phpMoAdmin(phpmoadmin.com) to connect it, it throws error like this:

PHP Fatal error: Uncaught exception ‘Predis\\Connection\\ConnectionException’ with message ‘Permission denied [tcp://127.0.0.1:6379]’

I checked the default dirs (/var/log/, /var/run/, /var/lib/ etc) for permission, also the owner/group, test without port 80, it seems work fine. But when in browser or curl to access, it failed.
Finally I got the answer from stackoverflow: Troubleshooting “Permission denied” when attempting to connect to Redis from PHP script: By default, SELinux does not allow Apache to make socket connections. To solve it, should run the following command first:

 sudo /usr/sbin/setsebool httpd_can_network_connect=1 
sudo  setsebool -P httpd_enable_homedirs 1

Here is a detailed explain:

httpd scripts by default are not allowed to connect out to the network.
This would prevent a hacker from breaking into you httpd server
and attacking other machines. If you need scripts to be able to
connect you can set the httpd_can_network_connect boolean on.

setsebool -P httpd_can_network_connect 1

I wrote the /usr/sbin/setsebool httpd_can_network_connect=1 in /etc/profile, so no worry for MongoDB and Redis-server connection anymore.
Actually I met this before, and run the same command to solve it, but forgot this time, so write here for retrieve. More info is available at: SELinux/apache.

CentOS: run crontab as root

CentOS: run crontab as root

Sometimes in a cron job, we have to write security directory such as /var/data/mysql/, in this case we need root permission.
There are several ways to run cron job by using root role. A simple solution is using ‘sudo’:
Normally as a normal user ‘demo’, I use sudo to borrow root permission by adding entry in /etc/sudoers with ‘nopasswd:all’. Similarly, in this case, we need to set sudo option in /etc/sudoers.
1. $ sudo vi /etc/sudoers:

Defaults    requiretty
Defaults:demo !requiretty

After the setting, we can sudo directly in crontab, without extra-declaration:
2. $ crontab -e

*/10 * * * * sudo /usr/bin/indexer -c $HOME/etc/file.conf --rotate delta --quiet

indexer will update /var/data/mysql directory every 10 minutes by using root permission. This works fine.

Perl: setup LD_LIBRARY_PATH

Perl setting LD_LIBRARY_PATH

When cpan installs Perl module with dynamic share object (.so) loading, it is probably with error like this:
Can’t load module.so: cannot open shared object file: No such file or directory. The error comes from /usr/lib64/perl5/DynaLoader.pm line 82:

if ($ldlibpthname_defined &&
    $ldlibpthname ne 'LD_LIBRARY_PATH' &&
    exists $ENV{LD_LIBRARY_PATH}) {
    push(@dl_library_path, split(/$pthsep/, $ENV{LD_LIBRARY_PATH}));
}

It indicates that perl can’t find the relative .so file, and probably the env variable LD_LIBRARY_PATH is not set.
The following are several ways to set LD_LIBRARY_PATH for Perl scripts:

  1. Add in Perl script itself:
    BEGIN {
     $ENV{LD_LIBRARY_PATH} = "/usr/local/lib";
    }
    # after setup the path, call Perl module which relys on the path:
    use Text::module;

    If problem, refer to this: Runtime Linker and LD_LIBRARY_PATH for proper description.

  2. Add in $HOME/.bash_profile
    This is local variable, only available for current login user. Edit $HOME/.bash_profile, adding the following line in the bottom:
    export LD_LIBRARY_PATH=/usr/local/lib
  3. Add in /etc/profile
    This is global variable, will effect all login users. Edit /etc/profile file, adding the following line in the bottom:
    export LD_LIBRARY_PATH=/usr/local/lib
  4. Operate in command line
    This is just for this section, will disappear when logout. In the command line:
    $ export LD_LIBRARY_PATH=/usr/local/lib
    $ perl -e ‘use Text:module; …’

It is always a good idea of using solution 1, put initial setting in perl’s BEGIN{} block.
In the crontab, for the variables inherit, we can do like this:
0 1 * * * (export LD_LIBRARY_PATH=/usr/local/lib; $HOME/perl_script >/dev/null 2>&1)

Install Coreseek-3.2.13 in CentOS-6.2

Coreseek-3.2.13 install in CentOS-6.2

It is never easy to install Coreseek fulltext-search source-codes. The Coreseek’s installation documents are not very clear, and its installation largely depends on environment settings. So it’s common having problems to install Coreseek.

After smoothly installing Sphinx 2.0.5, SCWS-1.2.0, I finally installed successfully Coreseek-3.2.13 in CentOS 6.2.
Here I summarized the steps.

1. pre-install support packages.

Read the pre-install requirements carefully and check what missed on the server.
I use ‘yum’ to install whatever I needed.

$ sudo yum install mysql-devel libxml2-devel expat-devel imkae gcc-c++

You can use ‘sudo yum install’ to install other packages such as ‘Development tools’, ‘php-devel’ etc if needed.

2. Dowload coreseek 3.2.13

Downlad Coreseek, which include 3-parts, and need to install 1 by 1:

$ wget http://www.coreseek.cn/uploads/csft/3.2/coreseek-3.2.13.tar.gz
$ tar xzvf coreseek-3.2.13.tar.gz
$ cd coreseek-3.2.13

3.Install mmseg part

$ cd mmseg-3.2.13

If you run ./bootstrap directly, the first command:
aclocal -I config
will fail:
config/sys_siglist.m4:20: warning: underquoted definition of SIC_VAR_SYS_SIGLIST

Here is 2 small tricks to fix it:
(a) edit bootstrap file, change shabang from #!/bin/sh to #!/bin/bash to inherit path and env variables.
(b) use root instead to execute.
$ sudo ./bootstrap

It should pass, then use ‘root’ permission to do configure & make:
$ sudo ./configure –prefix=/usr/local/mmseg
$ sudo make
$ sudo make install
Without ‘sudo’, there probably throw a lot of warnings and errors.

4. Install Coreseek part

$ cd csft-3.2.13/

Again, I change buildconf.sh’s shabang from #!/bin/sh to #!/bin/bash.

$ sudo bash buildconf.sh

$ ./configure –prefix=/usr/local/coreseek
–without-unixodbc –with-mmseg
–with-mmseg-includes=/usr/local/mmseg/include/mmseg/
–with-mmseg-libs=/usr/local/mmseg/lib/ –with-mysql

$ make; sudo make install

This way it compiled with mysql, and mmseg library.

5. Test by testpack

$ cd testpack/
$ locale
to check locale’s LANG’s setting, do the following setup:
$ export LANG=zh_CN.utf-8
$ export LC_ALL=zh_CN.utf-8

You may put these variables into /etc/profile.d/lang.sh or $HOME/.bash_profile or /etc/sysconfig/i18n or $HOME/.i18n.

$ /usr/local/mmseg/bin/mmseg -d /usr/local/mmseg/etc var/test/test.xml
The correct word segments display. That is cool.

From it’s README (http://www.coreseek.cn/products-install/install_on_bsd_linux/), it needs higher version of m4,automake,autoconf (in CentOS 5.5) which I don’t have.
My platform is CentOS 6.2(not CentOS 5.5), and the versions of these tools are lower than their requirements.
However, it still works.