How to install phantomjs and casperjs on bluehost linux vps

Last week I needed to automate an internet lookup in real time which required a several step login process (no API available). I was having trouble getting it to work with Curl. I had heard good things about CasperJS and PhantomJS so I figured I would try it out. I decided to use my own bluehost vps server to start with because it is always easier to develop a program which you have full control over.

CasperJS logo

I started with PhantomJS by itself, because I figured if I could get that working I didn’t need casper. However, after attempting the follow the installation instructions and using Phantom directly, I was getting errors that the client class was not found. I tried manually including the class file but that just led to more and more missing class and object errors because the whole library is setup to autoload things. After manually including the vendor/autoload.php file I at least didn’t get any more errors, but the simple examples were giving me responses of 0 so I decided I needed a different approach.

Installing CasperJS was relatively easy, but let me share the steps I followed to actually get phantom installed (it could be there is something I missed which prevented it from working by itself, but since I got casper working later I was satisfied):

  1. Login to the server through SSH (I use putty for this) as root (this is needed later. If you have a non privileged user who can login through ssh then you can start with that. Or if you’ve disabled root logins then login with the user who can become root)
  2. Become a non privileged user (type ‘sudo su [username]’, where username is the owner of your existing web files – the specific user here is important to avoid permission errors later).
  3. Create a directory for casper and phantom, like automation or browsercontrol, in a directory above an existing domain or subdomain so it’s not accessible in a browser (for security reasons)
  4. CD to the new directory and install composer there (even if composer is already a recognized command, do this anyway): curl -s http://getcomposer.org/installer | php
  5. Create a file in that directory called composer.json with these contents (this is straight from the installation guide):
    {
            "require": {
                "jonnyw/php-phantomjs": "4.*"
            },
            "config": {
                "bin-dir": "bin"
            },
            "scripts": {
                "post-install-cmd": [
                    "PhantomInstaller\\Installer::installPhantomJS"
                ],
                "post-update-cmd": [
                    "PhantomInstaller\\Installer::installPhantomJS"
                ]
            }
        }
    
  6. Try this command to install phantomjs:
    php composer.phar install
    
  7. If that doesn’t work (for example there is no bin folder created, and/or phantomjs is not created anywhere locally), then go to http://phantomjs.org/download.html and pick a link to manually download to your server (in your new folder, for example this is the command I used from my new directory):
    wget https://bitbucket.org/ariya/phantomjs/downloads/phantomjs-2.1.1-linux-x86_64.tar.bz2
    
  8. Extract the downloaded file (x means extract, j means it’s a bz2 file, and f means the file name is coming next):
    tar xjf phantomjs-2.1.1-linux-x86_64.tar.bz2
    
  9. Use this command to figure out what folders are currently in your path:
    echo $PATH
    
  10. Pick one like /usr/bin which is not for system programs (/bin and /sbin are, so don’t use those to avoid confusion later)
  11. Figure out the absolute path to phantomjs in your system now by finding it and then using pwd to get the path (likely ending with something like phantomjs-2.1.1-linux-x86_64/bin/phantomjs)
  12. Become root (if you logged in as root you can be root again by typing ‘exit’ at the prompt)
  13. Create a symbolic link to phantomjs inside the folder you picked in step 10 (like /usr/bin). Something like this:
    ln -sf /full/path/to/phantomjs /usr/bin/phantomjs
    
  14. Validate it worked by becoming the non-privileged user again and typing “phantomjs –version”. You should see a version number, not a complaint about no command found

Then for CasperJS:

  • Use slightly modified instructions from casperjs.org to install casper from git and put a link to it into your path (as the non privileged user, starting in the new folder you created):
    git clone git://github.com/casperjs/casperjs.git
    cd casperjs
    ln -sf `pwd`/bin/casperjs /usr/bin/casperjs
    

    If you picked a different folder in step 10, use it in the second part of the third command instead of /usr/bin.

  • Validate casper install by typing casperjs from the command line from any folder. You should see info about casperjs, not a complaint about a missing command.
  • In order to use casper within a PHP file, you will need to use the exec command (make sure that is allowed in your php.ini file). Here is a test php file you can use to make sure it is setup fully:

    if(function_exists('exec')) {
    	echo "exec is enabled<br>";
    } else echo "exec is not enabled<br>";
    exec("phantomjs --version", $outArr1);
    exec("casperjs", $outArr2);
    echo "<br>PhantomJS version output:<br>";
    print_r($outArr1);
    echo "<br>CasperJS output:<br>";
    print_r($outArr2);
    

    If you see a version number like 2.1.1 for phantom and a bunch of info about casper (after the CasperJS output line) you are good to go. The next step is to follow the casper instructions to create your javascript logic and then change your casper exec command to be something like this:

    exec("casperjs casperLogic.js --getParam1=\"This is passed to the script\" --getParam2=\"This is also passed to the script\"", $outArr);
    

    Happy automating!

    How to Add a Form in a WordPress Post

    This is a simple step by step guide to creating a form that will capture data and store it in a mysql database on your server from within a wordpress post

    Note: the below example is a live, working form on this page

    Step 1:

    Create a simple form for a petition, contact request, or registration:

    Name:
    Address:
    City:
    State:
    Email:
    Comment:

    Here is the html code for it:

    <table>
    <tr><th>Name:</th><td><input id="userName" /></td></tr>
    <tr><th>Address:</th><td> <input id="userAddress" /></td></tr>
    <tr><th>City:</th><td> <input id="userCity" /></td></tr>
    <tr><th>State:</th><td> <input id="userState> /></td></tr>
    <tr><th>Zip:</th><td> <input id="userZip" /></td></tr>
    <tr><th>Email:</th><td> <input id="userEmail" /></td></tr>
    <tr><th>Comment:</th><td> <textarea id="userComment" ></textarea></td></tr>
    <tr><th><td colspan=2><button type="button" onclick="saveUser();">Save</button></td></tr>
    </table>
    

    Step 2:

    Create a javascript function to process the form and send it to the server. The following is a javascript function added to the page using the CSS & Javascript Toolbox plugin (it uses jquery because wordpress already has that available):

    //Process the form and save the data record
    function saveUser() {
      //First gather the form parameters and make sure name and email at least are populated
      var name = jQuery("#userName").val();
      var address = jQuery("#userAddress").val();
      var city = jQuery("#userCity").val();
      var state = jQuery("#userState").val();
      var zip = jQuery("#userZip").val();
      var email = jQuery("#userEmail").val();
      var comment = jQuery("#userComment").val();
      if (name.length<1 || email.length<1) {
        alert("Please at least enter your name and email.");
        return false;
      } else {
        //Now send the data to a server side function to really validate it and save it.
        jQuery.ajax({
          type: "POST",
          url: "/ajax/saveUser.php",
          data: { name:name,address:address,city:city,state:state,zip:zip,email:email,comment:comment }
        }).done(function( results ) {
          if(results.length<1){ // network error
            alert("There was a network error, please try again or contact support and tell them what you are trying to do.");
          } else { // this is a successful interaction
            var resultObj = jQuery.parseJSON(results);
                    
            if (resultObj.errorMsg.length>0) {  
              alert(resultObj.errorMsg);
            } else {
        	  //Record save successful
              alert("Thanks for your information, it was saved successfully!");
              //Show the user what they have entered:
              jQuery("#userList").html(resultObj.userList);
            }
          }
        });
      }
    }
    

    Step 3:

    Make sure you have a database table ready to store the information. Below is a simple table used to store the info in this example:

    CREATE TABLE IF NOT EXISTS `user_info` (
      `userID` int(11) NOT NULL AUTO_INCREMENT,
      `userName` varchar(80) DEFAULT NULL,
      `address` varchar(100) DEFAULT NULL,
      `city` varchar(80) DEFAULT NULL,
      `state` varchar(40) DEFAULT NULL,
      `zip` varchar(5) DEFAULT NULL,
      `email` varchar(80) DEFAULT NULL,
      `comment` text DEFAULT NULL,
      `userIP` VARCHAR( 30 ) DEFAULT NULL,
      `dateAdded` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP,
      PRIMARY KEY (`userID`),
      KEY `userIP` (`userIP`)
    ) ENGINE=InnoDB DEFAULT CHARSET=latin1 AUTO_INCREMENT=1 ;
    

    Step 4:

    Create the server side code to accept the form data, really validate it, and put it in the database or send back an error. Here is the contents of a server side code which will do all that for the example data above. Since this form is “in the wild”, I’m capturing the IP address because I will only show results to people that they entered themselves:

    /*
     * File: saveUser.php
     * Description: This file takes the passed in user information and validates it before
     * 	saving it to the database and returning content to show on the page. 
     * Inputs (in POST): 
     * 	userName, address, city, state, zip, email, comment
     * Outputs:
     * 	either an error or nothing
     */
    include("db.php"); // define your database credentials in another file
    
    #connect to the database
    $db = mysqli_connect($servername, $username, $password, $dbname);
    mysqli_select_db($db,$dbname) or die('Could not select database');
    
    #Get the passed in parameters into local variables, escaped for database input
    $userName = empty($_POST['userName'])?'':addslashes($_POST['userName']);
    $address = empty($_POST['address'])?'':addslashes($_POST['address']);
    $city = empty($_POST['city'])?'':addslashes($_POST['city']);
    $state = empty($_POST['state'])?'':addslashes($_POST['state']);
    //only accept 5 numbers for zip
    $zip = empty($_POST['zip'])?'':substr(preg_replace('/\D/','',$_POST['zip']),0,5);
    $email = empty($_POST['email'])?'':addslashes($_POST['email']);
    $comment = empty($_POST['comment'])?'':addslashes($_POST['comment']);
    $userIP = $_SERVER['REMOTE_ADDR'];
    
    #This is an array used for gathering all the outputs of this file
    $jsonObj = array();
    $jsonObj['errorMsg'] = "";
    $debug=false;
    
    #Validate inputs
    if (empty($userName) or empty($email)) {
    	$jsonObj['errorMsg'] = "Please at least enter your name and email.";
    } else if (strpos($email,'@')===false) {
    	//there are many more validations that can be made for emails but this is a start
    	$jsonObj['errorMsg'] = "Please enter a valid email.";
    }
    
    #Enter the data record
    if (empty($jsonObj['errorMsg'])) {	
    	$sql = "insert into user_info (userName,address,city,state,zip,email,comment,userIP) 
    			values
    			('".$userName."','".$address."','".$city."','".$state."','".$zip."','".$email."',
    			'".$comment."','".$userIP."')";
    	if ($debug) echo "about to run $sql ";
    	mysqli_query($db, $sql); $err = mysqli_error($db); if (!empty($err)) $jsonObj['errorMsg'] = $err;
    }
    
    #Now get the list of data entered by this IP, without any script tags to prevent XSS:
    $jsonObj['userList']='';
    if (empty($jsonObj['errorMsg'])) {	
    	$sql = "select userName,city,state from user_info where userIP='".$userIP."'";
    	$rs = mysqli_query($db, $sql); $err = mysqli_error($db); if (!empty($err)) $jsonObj['errorMsg'] = $err;
    	while($row = mysqli_fetch_assoc($rs)) {
    		$jsonObj['userList'].= "<tr><td>".strip_tags($row['userName'])."</td><td>".strip_tags($row['city']).
    				"</td><td>".strip_tags($row['state'])."</td></tr>";
    	}
    }
    
    #Now send back the data in json format
    echo json_encode($jsonObj);
    

    Step 5:

    Do something with the results! You could export them, feed them into another system using an API, send them in an email to someone, or just display them like so:

    Name City State
    You have not entered any user records yet, try it out!

    Google Maps API Version 3 Geocoding Example

    Today I realized that my location markers were not being displayed in all the google maps implementations I had put into place online, and after digging into it (trying to figure out what a code 610 response meant) I realized that I was using version 2 of the google maps api for retrieving latitude and longitude based on a given address (ie “geocoding”).

    My actual maps were being displayed using version 3 of the API but the example I found a couple years ago and followed for geocoding used version 2, which was deprecated and will no longer be supported sometime in 2013 (I’ve heard both March and September, but since my code isn’t working in April I suppose the March date was more accurate).

    Since I wasn’t able to find an example for just what I needed (getting the latitude and longitude based on an address, so I could show markers on a google map) I read through the docs and updated my code.  See below for my implementation (I use this as an included file wherever I need to pull latitude/longitude, it could be put into a function if you want). The address, city, state, and zip should be stored in the $address, $city, $state, and $zip params, and I am also saving the latitude and longitude in both a local database and a markers array to be used in a google map later:

    /*
     * File: incGetGeoCodes.php
     * Description: this file pulls the lat/long coordinates using Version 3 of the google maps api
     * 	The key parameter caused it to fail so I removed that and it works, but we may be under a 
     * 	lower daily usage limit because of that
     */
    // Initialize delay in geocode speed
    $delay = 0;
    $base_url="http://maps.googleapis.com/maps/api/geocode/json?sensor=false";
    
    $geoaddress = $address . ", " . $city . ", " . $state . " " . $zip;
    $request_url = $base_url . "&amp;address=" . urlencode($geoaddress);
    $json=file_get_contents($request_url);
    $resultArr=json_decode($json,true);
    
    $error='';
    $status = $resultArr['status'];
    if (strcmp($status, "OK") == 0) {
    	// Successful geocode
    	$markers[$shop_id]['lat'] = $resultArr['results'][0]['geometry']['location']['lat'];
    	$markers[$shop_id]['long'] = $resultArr['results'][0]['geometry']['location']['lng'];
    	$markers[$shop_id]['html'] = $name."<br>".$address."<br>".$city." ".$state." ".$zip."<br>".$phone;
    	
    	#Now update the db so we don't have to pull this again
    	$query = "update entities ".
    		"set shop_latitude=".addslashes($markers[$shop_id]['lat']).", ".
    		"shop_longitude=".addslashes($markers[$shop_id]['long'])." ".
    		"where EntityID=".intval($EntityID);
    	mysql_query($query); $err=mysql_error(); if (!empty($err)) echo "query:$query, error: ".$err."<br>";
    } else if (strcmp($status, "620") == 0) {
    	// sent geocodes too fast
    	$delay += 100000;
    } else {
    	// failure to geocode
    	$error .= urlencode("Address " . $geoaddress . " failed to be geocoded. ");
    	$error .= urlencode("Received status " . $status . "%0D%0A");
    }
    usleep($delay);
    

    I was surprised to see no mention of an API key parameter to be used, if someone knows if that is an option (to increase the daily quota of geocoding api calls that can be made) please leave a comment and let me know. I’m just happy to get it working again for now. 🙂

    How to integrate existing website into Amazon Simple Email Service

    I was preparing to integrate a client’s business to send emails through AWeber last week when I realized that their api does not support sending transactional emails to one or more people on an as needed basis. I confirmed with AWeber’s tech support that their API is read only, it does not allow the sending of emails at all (they have a web interface for sending emails). I asked them what they would use in my situation and they said that the other big newsletter names I had heard of before (MailChimp, Constant Contact, etc.) also only supported newsletter type messages.

    What my client needed was a more robust email system because every week they had situations where people were not getting emails (sent from their own server using the php mail command). I recommended AWeber because I knew their deliverability was very high and they would keep up with any changes in email standards. I figured since they had an API that I could specify email addresses and email contents to send using it but after looking for that functionality I came up empty handed.

    The Aweber tech I spoke to mentioned the possibility of using SalesForce for this type of account message emailing, but I knew that would be overkill and overpriced for just sending emails. After a quick search I was happy to find out that Amazon provides an email service called “Simple Email Service” (SES) that allows for a certain number of emails for free if you already have an Amazon Web Services (AWS) account. Since my client had signed up for the Amazon S3 service (a storage solution) a few months prior in order to save backups of their weblogs they already had an AWS account.

    After reading a few documents about different ways to use the Amazon Simple Email Service I decided that it would be simplest for me to integrate using an authenticated SMTP connection. Since there were only half a dozen files that use the mail command (found by using ‘grep “mail(” *php’ and ‘grep “mail(” *\*php’ and so on in the webroot), I only needed to update those files after getting the Pear modules installed.

    I started using just the pear Mail module but then when I tried to send html emails they showed up as source code and not rendered, so I added the Mail_Mime files too. The way I did it was to first try installing it using “pear install Mail” (and pear install Mail_Mime) as root, but after a bunch of trouble with include paths I ended up downloading the tarballs to the server using wget, then extracting them into a subdirectory under the webroot (which I later protected using htaccess to deny all connections directly to the files). Next I tried including the main Mail.php file with error printing on and updated several of the pear files to refer to the right relative path for inclusion. I did the same thing with the Mail/mime.php file, adjusting paths as needed until the errors were all gone.

    I had a common included file at the top of each of my php files so inside that common file I included the pear files and defined a few constants for connecting to the smtp server at amazon (pear is the name of the folder in my webroot where I put the pear files):

    #Show errors - after I got the paths right I commented this section
    error_reporting(E_ALL);
    ini_set('display_errors', true);
    ini_set('html_errors', true);
    
    #Pear email modules
    require_once "pear/Mail.php";
    require_once "pear/Mail/mime.php";
    define("MAILHOST","ssl://email-smtp.us-east-1.amazonaws.com");
    define("MAILPORT","465");
    define("MAILUSER","myAmazonSMTPUserNameHere");
    define("MAILPWD","myAmazonSMTPPasswordHere");
    

    Then in each file where I wanted to send email, I used this framework to do it:

    # Constructing the email
    $sender = "Sender Name <senderName@domain.com>";                              
    $recipient = "recipientName@domain.com;                           // The Recipients name and email address
    $text = "this will be sent as text;                                  // Text version of the email
    $html = "<h1>This will be rendered as html</h1>";  // HTML version of the email
    $crlf = "\n";
     		        
    $headers = array(
            'From'          => $sender,
            'To'          => $recipient,
            'Subject'       => $subject
            );
    
    # Creating the Mime message
    $mime = new Mail_mime($crlf);
    
    # Setting the body of the email
    if (!empty($text)) {
            $mime->setTXTBody($text);
    } else if (!empty($html)){
            $mime->setHTMLBody($html);
    }
    
    #Get the header and body into the right format
    $body = $mime->get();
    $headers = $mime->headers($headers);
    $headers['From'] = $sender;  //I heard some people had trouble with this header getting messed up
    
    #Setup the connection parameters to connect to Amazon SES
    $smtp_params["host"]     = MAILHOST;
    $smtp_params["port"]     = MAILPORT;
    $smtp_params["auth"]     = true;
    $smtp_params["username"] = MAILUSER;
    $smtp_params["password"] = MAILPWD;
     
    # Sending the email using smtp
    $mail =&amp;amp; Mail::factory("smtp", $smtp_params);
    $result = $mail->send($recipient, $headers, $body);		
    
    #Below is only used for debugging until you get it working
    if (PEAR::isError($result)) {
       echo("<p>" . $result->getMessage() . "</p>");
    } else {
       echo("<p>Message successfully sent!</p>");
    }
    

    Caveat

    Amazon doesn’t put your emails into a queue if you send them too fast, so in order to stay under their sending limits when sending batches of messages you can use the php usleep command to delay execution. I found that this delay didn’t actually work until I added “set_time_limit(0);” to the top of the file sending the batch of emails, however. Test everything, different server environments will respond differently (just like Browsers are like Churches). I used an echo date(‘h:i:s’) command between delays to see whether the delay worked or not.

    How to use Ajax in a WordPress Post

    I recently put together an Ajax autocomplete example using PHP and Mysql so I could post it on my blog, but after getting it working outside wordpress on an independent page I had trouble getting it into wordpress.

    I had used plugins in the past which allowed for php code to be used with shortcodes, but inline javascript calls are not something I had gotten past the wordpress editor before. The php code used for retrieving data from the database is in a separate file so no shortcode is needed for that portion.

    The inclusion of the javascript autosuggest functions is something I first tried putting in a code block in the wordpress post itself as a link to the external file (autosuggest.js), but unfortunately that didn’t work. Then I saw an article where someone suggested putting the javascript in the theme file header.php and surround it with a php if statement checking for the specific post you want it to appear on. I tried that but later removed it because I wanted my ajax to work on the index page, archive pages, and category/tag/author pages – not just the “single post” page.

    Then I went back to the plugin I had installed (CSS & JavaScript Toolbox) for css and javascript and tried putting the entire contents of the autosuggest.js file into one javascript block and installing that in the footer. That worked perfectly so I did the same thing for the css file using the same plugin.

    The next challenge was to add the inline JavaScript (ajax) function calls from within the html code for the sample input box in the wordpress post. When I tried just putting it all into the html editor for wordpress the javascript was removed. This was not just a snippet of complete javascript or css though, so I couldn’t just use the same plugin to make another code block and put it in the header or footer.

    Since PHP is executed as the page loads, I figured I could put my auto-complete input box with ajax call into a php shortcode and put the shortcode where I wanted the input box to be in my html code. That way wordpress would not filter out the javascript when saving the post contents to the database because it would be hidden behind a shortcode which would be replaced during page loading.

    This worked beatifully, including the inline javascript by putting it inside a php shortcode. I used the Shortcode Exec PHP plugin for this, which is very easy to work with.

    Allow Email as Username within WP-Members Plugin

    Today I figured out how to let people use their email to login instead of their username to the WordPress driven site called Jacked Pack. I had already installed the WP-Members plugin in order to add custom registration fields to the WordPress account page (for the “HUGEness profile”), but when I tried installing the WP Email Login plugin it didn’t have any effect. I figured this was because the WP-Members plugin already supercedes the regular login functionality, so I asked Chad Butler (who created the WP-Members plugin) about the issue.

    Fortunately Chad had already created a way for people to add plugins to his plugin, so that’s what I ended up doing in this case. I followed his instructions by creating a file called wp-members-pluggable.php in my /wp-content/plugins directory and copying the wpmem-login function into it (I found that function in the wp-members-core.php file in the wp-members plugin folder). Then I added my own simple function (which was taken from the WP Email Login plugin) to get the username from an email address in case someone enters an email in the username field. Then I added just one line to the wpmem-login function which calls my function. Once I got that working, I copied the wpmem_inc_login function (which I found in the wp-members-dialogs.php file in the wp-members plugin folder) into my wp-members-pluggable.php file and changed one line to tell people they can use Username or Email in the first field.

    See below for my complete plugin for the WP-Members WordPress Plugin, or download it here:

    <?php
    /**
     * WP-Members Pluggable Functions
     *
     * These functions replace those in the wp-members plugin
     * 
     */
    
    if( ! function_exists( 'wpmem_login' ) ):
    /**
     * Logs in the user
     *
     * Logs in the the user using wp_signon (since 2.5.2). If login 
     * is successful, it redirects and exits; otherwise "loginfailed"
     * is returned.
     *
     * @since 0.1
     *
     * @uses apply_filters Calls 'wpmem_login_redirect' hook to get $redirect_to
     *
     * @uses wp_signon
     * @uses wp_redirect Redirects to $redirect_to if login is successful
     * @return string Returns "loginfailed" if the login fails
     */
    function wpmem_login()
    {
    	if( isset( $_POST['redirect_to'] ) ) {
    		$redirect_to = $_POST['redirect_to'];
    	} else {
    		$redirect_to = $_SERVER['PHP_SELF'];
    	}
    	
    	$redirect_to = apply_filters( 'wpmem_login_redirect', $redirect_to );
    
    	if( isset( $_POST['rememberme'] ) == 'forever' ) {
    		$rememberme = true;
    	} else {
    		$rememberme = false;
    	}
    
    	if( $_POST['log'] &amp;amp;amp;amp;amp;&amp;amp;amp;amp;amp; $_POST['pwd'] ) {
    		
    		$user_login = sanitize_user( $_POST['log'] );
    
    		$user_login = wpmem_login_check_for_email($user_login);
    		
    		$creds = array();
    		$creds['user_login']    = $user_login;
    		$creds['user_password'] = $_POST['pwd'];
    		$creds['remember']      = $rememberme;
    		
    		$user = wp_signon( $creds, false );
    
    		if( ! is_wp_error( $user ) ) {
    			if( ! $using_cookie )
    				wp_setcookie( $user_login, $user_pass, false, '', '', $rememberme );
    			wp_redirect( $redirect_to );
    			exit();
    		} else {
    			return "loginfailed";
    		}
    	
    	} else {
    		//login failed
    		return "loginfailed";
    	}	
    
    } // end of login function
    endif;
    
    
    if ( ! function_exists( 'wpmem_login_check_for_email' ) ):
    /**
     * Get Username from Email
     *
     * Takes the username and checks if there is a user with an email that matches.
     * 	If there is, the username associated with the email is returned.
     * 	Otherwise the username is returned as it was passed in.
     *
     * @uses get_user_by
     * @return string Returns username associated with email or the same
     * 	username that was passed in
     */
    function wpmem_login_check_for_email($username) {
    	if ( !empty( $username ) )
    		$user = get_user_by( 'email', $username );
    	if ( isset( $user->user_login, $user ) )
    		$username = $user->user_login;
    	return $username;
    } // end of check for email in the username
    endif;
    
    
    
    if ( ! function_exists( 'wpmem_inc_login' ) ):
    /**
     * Login Dialog
     *
     * Loads the login form for user login
     *
     * @since 1.8
     *
     * @uses apply_filters Calls wpmem_restricted_msg filters message content
     * @uses wpmem_login_form()
     *
     * @param string $page
     * @return string the generated html for the login form
     */
    function wpmem_inc_login( $page="page" )
    { 	
    	global $wpmem_regchk;
    
    	$str = '';
    	$arr = get_option( 'wpmembers_dialogs' );
    
    	if( $page == "page" ){
    	     if( $wpmem_regchk!="success" ){
    		
    			// this shown above blocked content
    			$str = '<p>' . stripslashes($arr[0]) . '</p>';
    			
    			// filter blocked content message
    			$str = apply_filters( 'wpmem_restricted_msg', $str );
    
    		} 	
    	} 
    
        //$arr = array( __( 'Existing users Login', 'wp-members' ), __( 'Username', 'wp-members' ), 'text', 'log', __( 'Password', 'wp-members' ), 'password', 'pwd', 'login', __( 'Login', 'wp-members' ), 'username', 'password' );
    	$arr = array( __( 'Existing users Login', 'wp-members' ), __( 'Username or Email', 'wp-members' ), 'text', 'log', __( 'Password', 'wp-members' ), 'password', 'pwd', 'login', __( 'Login', 'wp-members' ), 'username', 'password' );
    	
    	$str = $str . wpmem_login_form( $page, $arr );
    	return $str;
    }
    endif;
    ?>
    

    Update July 5th, 2013

    I was asked about the registration function as well (see comments from Brian Weiss below) and although I haven’t made a plugin for the WP-Members registration function I have done auto registrations in a separate file. I tried posting code in a comment and it didn’t turn out well, so here is the relevant code I used (in a separate php file I created in the wordpress directory) to auto create wordpress accounts based on their email and firstname (you could do it without using their name if you want, since the users will never need to know their username). If someone puts this into a plugin for the WP-Members registration function please let me know and I’ll link to it so others who want to do that can see the solution.

    #Initial try at an unclaimed username
    $username = $firstName.rand(1,1000);
    
    #Now create a WP account
    include('wp-config.php');
    
    // Initialize connection and select database
    mysql_connect(DB_HOST, DB_USER, DB_PASSWORD)
        or die('Could not connect: ' . mysql_error());
    //echo 'Connected successfully';
    mysql_select_db(DB_NAME) or die('Could not select database');
    //echo "connected to db<br>";
    
    #WP Username has to be unique
    while (username_exists( $username )) {
    	$username = $firstName.rand(1,1000);
    }
    $random_password = wp_generate_password( $length=12, $include_standard_special_chars=false );
    //echo "about to create user $username with pwd $random_password and email $email<br>";
    $user_id = wp_create_user( $username, $random_password, $email );
    if (is_int($user_id)) {
    	//echo "new user id is $user_id<br>";
    } else {
        if(is_wp_error($user_id)) { echo $user_id->get_error_message(); }
        exit;
    }
    // then send a welcome email and/or redirect wherever you want...
    

    Simple Ajax Auto-Complete Example

    In a previous post I wrote about how I learned ajax quickly, but recently I learned that the main link I learned from is no longer valid. So I decided to make a simple example I could host myself. Along the way I learned that it is not trivial to include ajax within a wordpress page, but I wrote another post about how I figured that out. 🙂

    The input box below is enabled with auto-complete using a simple ajax example. The javascript can be seen here, the css can be seen here, and the php ajax function can be seen here. The input box below calls the getScriptPage javascript function if there are at least 2 characters present. That javascript function calls the php ajax function using a server request. The text sent back from the server by the php file is then handled by the handleHttpResponse javascript function, which just populates a hidden div with choices. The getScriptPage function makes that hidden div (with an id of “box”) visible if the response from the php ajax call is not empty.

    The database table being queried has 1000 rows in it, in the following format: name100 (where 100 is the number of the record, 1-1000). So if you start typing “name” in the box, after na you will start seeing 5 suggestions below. If you keep typing the suggestions will be updated as you type.

    HTML code for input box

    <div id="headersearch">
    	<div id="searchbar">
    		<div class="ajax-div">
    			<div class="input-div">
    			<input type="text" autocomplete="off" onKeyUp="if(this.value.length>1) 
    				getScriptPage('box','searchTerm','http://www.wilycode.com/'); else box(0);" 
    				name="searchTerm" value="Enter Name Here" onClick="if(this.value.substr(0,6)=='Enter ') this.value=''"
    				id="searchTerm" onBlur="if(this.value.length<1) this.value='Enter Name Here'">
    			</div>
    			<div id="box"></div>
    		</div>
    	</div>
    </div>
    

    April 24th 2014 Update:

    I was asked to add arrow navigation to an autosuggestion box I created this week using the method described above, and found that there are easier ways to do this. I found that jquery UI has an autocomplete widget which can be used to enable arrow key navigation among many other features. However I had trouble getting the examples at api.jqueryui.com/autocomplete/#entry-examples to work, so I did some more searching and found a step by step implementation using a php file for the source element of the autocomplete function which I was able to get working pretty easily, at www.pontikis.net/blog/jquery-ui-autocomplete-step-by-step.

    How to Easily Prepare Database Import Using Perl

    I’m revisiting this topic because in my last post about translating Excel data into a text file for database import I used a php script and yet still had some manual steps (the one that bugged me most was adding custom line termination characters for multiline data). I mentioned the idea that if I wanted to put more time in I could use a Perl script to do the whole process. Well my next set of data I needed to import was almost 700 records spanning almost ten times as many lines in the tab delimited text file I exported from Excel (due to the description fields having newline characters). So here’s my latest solution:

    1. First I added a numeric column in the left-most position (within excel) and filled it with incremental integers.
    2. Then I exported the spreadsheet as tab delimited text
    3. Then I wrote a perl script to loop through the file and add line termination characters before each new record, this time I had to use double equal signs (==) because the data contained single equal signs
    4. I also found a perl module similar to the php htmlentities() function which actually worked for me, as opposed to the php function which I abandoned in favor of hard-coded string replacement commands
    5. Then I ran the perl script on my tab delimited text file to create a tab delimited file with double equal signs as the line terminator and all the funny characters encoded so they would show up correctly in a browser
    6. Finally I imported my file using the load data infile option in the phpmyadmin interface, specifying the delimiter and line termination as \t and == respectively.
    7. Then I looked at a webpage which pulled that data and it looked like it did in the excel file!

    Here is the code for the perl script I created:

    #!/usr/bin/perl
    # Filename: prepareTextImport.pl - this script will take a list of data 
    #	including multiline data where the first field is a line count.
    #	It will add == before each line count to enable mysql importing and 
    #	will encode html entities and convert newline characters to <br>
    use HTML::Entities;
    
    my $input = $ARGV[0]; #this is the file to parse
    my $output = $ARGV[1]; #this is the file to create
    my $lineTerminator = "==";
    my $count=1;
    my $line = '';
    open (IN,$input) or die "can't open file $input\n";  
    open (OUT, ">$output") or die "Can't open outfile : $output\n";
    while (<IN>) {	# each line of the current input file
    	chomp;
    	$line .= "<br>"; # include this here so the <br> will be encoded too
    	$line = encode_entities($_); #This is the function which encodes html symbols
    	# Check if this is the beginning of a new record.  If so then add the line terminator
    	if ($line =~ /^$count/) {
    		$count++;
    		$line = $lineTerminator.$line;
    	}
    	print OUT $line;
    }
    close IN;
    close OUT;
    print "there were $count records processed.\n";
    exit;
    

    The syntax for running the script (assuming perl is installed on your system – I use activeperl on my PC which is free):

    perl prepareTextImport.pl inputFileName.txt outputFileName.txt
    

    How to Convert Excel Spreadsheets to SQL Statements

    Last week I told a client he could send me a file in excel and I could import that into the mysql database directly. When I got the file I learned that it wasn’t quite that easy. I had planned to just save it as tab delimited text or CSV, but there was a “description” field which was copied from various webpages (by copying the rendered text in the browser window).

    Within the description field there were trademark symbols, copyright symbols, slanted quotes, triple dots, non-standard dash marks, bullets, and even a cross symbol (or dagger) used for footnotes. In addition, there were several paragraphs (including multiple line breaks) in each description field.

    The first challenge to overcome was getting the database to handle line breaks properly so I could import the data. I figured if I added a special character before each new record then I could cause the database to ignore the newline characters, but when I tried pipe (|) and three pipes (|||) neither of those worked (I only got one record imported each time). It finally worked for me when I added an equal sign (=) before each new record (after checking to make sure there were no equal signs in the text – if there had been it should have worked by escaping them with a backslash and telling mysql that is the escape character).

    In order to insert the equal signs I first saved the excel sheet as tab delimited text, then opened it in notepad++ and used their regular expression search and replace tool to do the insertions quickly. Since I had numbers in the first field of my data, I searched for ^(\d) and replaced it with =\1. If I wanted to spend more time on it I could have made a perl script to go straight from the .txt file to sql statements, but I already had a php file to create sql statements so I wanted to use that.

    Once I had the equal signs in place, I used a load data infile command (actually created by phpmyadmin, I just had to make the selections – the local keyword was necessary for permission requirements) like this:

    LOAD DATA LOCAL INFILE 'path-to-text-file-here' INTO TABLE 'myTableName' FIELDS TERMINATED BY '\t' ENCLOSED BY '"' ESCAPED BY '\\' LINES TERMINATED BY '=';
    

    I was happy to get the lines imported, but the line breaks were lost and special characters showed up as question marks on the webpage where I displayed the data. So then I went back to my php script used for going from access to mysql from a few years ago and made a few modifications. I first tried using the php htmlentities() command to convert the symbols back to html code but for some reason that didn’t work for me. So through an iterative process I added replace commands for each symbol I found. I wasn’t trying to create all the symbols but to preserve the essence of the text. If you want to convert all the symbols you find you can look at a w2schools reference symbol page for a list – there is also an entity list at w3schools, if you don’t find your symbol in the first list. I also wanted to preserve the newlines so I used the php command nl2br and that worked great. Here is the final code I added for processing each field:

    $data_value = trim($data_value);
    $data_value = trim($data_value,'"');
    if (empty($data_value)) continue;
    $data_value = str_replace('™','&amp;amp;trade;',$data_value);
    $data_value = str_replace('®','&amp;amp;reg;',$data_value);
    $data_value = str_replace('’','\'',$data_value);
    $data_value = str_replace('…','...',$data_value);
    $data_value = str_replace('•','-',$data_value);
    $data_value = str_replace('–','-',$data_value);
    $data_value = str_replace('†','*',$data_value);
    $data_value = str_replace('“','"',$data_value);
    $data_value = str_replace('”','"',$data_value);
    $data_value = str_replace('—','-',$data_value);
    $data_value = nl2br($data_value);
    

    I also couldn’t just copy the sql queries from the webpage because all the html code was converted back to symbols, so I added this line to write the queries directly to a local file:

    file_put_contents("queries.txt",$myQueries);
    

    Then I could open the queries.txt file and have a bunch of sql queries which I could run to populate my database table in mysql. My updated text to sql php script can be downloaded and used freely.

    Update: I put the time into creating a perl script to make this easier, so take a look at that if the above process is too much work for you. 🙂

    How to Add the Sidebar to WordPress Twenty Eleven Theme Single Posts

    This is my first post since splitting off my technical posts into a new blog focused in that area. Part of the process was setting up a new wordpress blog, and of course I wanted to try a new theme, the Twenty Eleven theme. After I began importing a few posts I realized that there was no simple option to add the sidebar to the single post view. I tried messing with the CSS for a few minutes and then realized that there must be plenty of answers from people about this issue already.

    I found one that looked promising at http://tricksmommy.com/2011/07/08/add-sidebar-in-single-post-pages-of-twenty-eleven-wordpress-theme/. Unfortunately after following the detailed instructions there the single post page had the sidebar added but the main index page was shifted off to the right. I couldn’t figure out the exact reason why, but the instructions mentioned copying css from another view so I figured it must be a different setup or something. I chose the left sidebar on my site whereas they used the default with the sidebar on the right, so that might be why it didn’t work for me. After removing those changes I came up with a simpler solution (step 1 is the same as the site mentioned above, step 2 is where we differ):

    1. Go the the Theme Editor and select the Single Posts(single.php). Search for the following line:
      <?php get_footer(); ?>
      

      Just above it place the following code:

      <?php get_sidebar(); ?>
      
    2. In the same file, look for the following code:
      <?php get_template_part( 'content', 'single' ); ?>
      					
      <?php comments_template( '', true ); ?>
      

      You’ll want to add another div around the above two lines and give it a new class name so you can set the positioning, like this:

      <div class="singlePostWrapper">
      <?php get_template_part( 'content', 'single' ); ?>
      					
      <?php comments_template( '', true ); ?>
      </div>
      
    3. Next you’ll open the style.css and add a simple definition for the new css class you created above as a wrapper around the post and comments on the single post pages. You can add this at the end of the file:
      /*Allow room for sidebar in single posts*/
      .singlePostWrapper {
        position:relative;
        left:25%;
        width:75%;
      }
      

      Note that the above class definition works well for the sidebar on the left, but if you chose to put the sidebar on the right in the theme settings you can change the word ‘left’ above to ‘right’ and it should work fine.