EllisLab text mark
Advanced Search
1 of 3
1
   
CI Proxy Library - Browser simulator (REST, Cookies and Proxy Support, PURE PHP) [NEW] HTTP Authorization
Posted: 12 April 2011 07:48 AM
Avatar
Joined: 2010-12-20
1591 posts

Hi all.

It’s been a while, since i use CI in my (commercial) project. What i like from it, beside other framework (i use several framework, i have 2 favorite : Kohana and CI smile ), its had enough helper and library to make developer life easier.

When i develop some project, i need to have a simple way to call another controller, or even an external resource (ex. api from google, or iptocountry@sourceforge). I create a simple cURL class to do that. This is the (almost) stable version of that, this enough for my needs. I’ll take it to higher level, as soon as i had a long holiday from my work.

It have several features, which might help you during development (or even in live web) process.

Here’s the way you use it

// Simple way to use this library
// In any controller, put this line...
...
$this->load->library('proxy');
$this->proxy->site('http://codeigniter.com',TRUE);
...

// Above example will give you rendered page of CodeIgniter site, if you didn't want
// to render it directly, or it was a json which you want to save to var, simply do this...
...
$this->load->library('proxy');
$json_var $this->proxy->site('http://ip2country.sourceforge.net/ip2c.php?ip='.$ip.'&format=JSON');
...

// To call your controller (maybe in some situation, you need it instead use redirect() function)
...
$this->load->library('proxy');
$this->proxy->controller('your_target_controller_name/target_function/some_id',TRUE);
... 

Grab it! And i’ll be here, if you had some feedback, bug report or some trouble with it.

Cheers!
Toopay

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 12 April 2011 01:42 PM   [ # 1 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

Well, because this library’s main purpose is just to fetch/send simple http get request, one feature from this library, which distinguish this library with (*sigh) other CI curl library is : It can maintain image url from targeted site (see the different, in attachment image below. For comparing, i use phil’s curl library to generate failed rendering page). It maintain anchor tag and form url too.

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 15 April 2011 05:00 PM   [ # 2 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

Uh-Oh! Updated to V 1.0.1

MORE FEATURES(All old feature still there):

1. Get Full HTTP Header.
2. Set Proxy Call.
3. Set Delay HTTP Call.
4. Set user agent.
5. Internal cache (using gzip).
6. Persistent call (processing redirect, either from header or meta)
7. NO CURL OR OTHER FANCY STUFF DEPEDENCIES! PURE PHP.
8. Cookie support.
9. Log and error flag.

$this->load->library('proxy');
//Set proxy call
$this->proxy->set_proxy('proxy_host',80,'username','password');
$this->proxy->site('twitter.com',TRUE);
//Get rendered HTTP header, for use it as variable, 
// put FALSE on second passed param or simple let it blank
$this->proxy->head('codeigniter.com',TRUE);
//Set user agent and delay
$this->proxy->set_delay(5);
$this->proxy->set_useragent($this->session->userdata('user_agent'));
$this->proxy->site('twitter.com',TRUE); 

Test or play with it, download Proxy Library on wiki

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 15 April 2011 05:18 PM   [ # 3 ]   [ Rating: 0 ]
Joined: 2009-01-21
48 posts

Very, very nice!
When you call another controller (in the same site/server), the delay is big?

 
Posted: 15 April 2011 05:21 PM   [ # 4 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

No! It fast(test it).

The delay is optional. You can use set_delay() function, if you made more than one site(chaining call).

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 16 April 2011 01:50 AM   [ # 5 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

V 1.0.2
cleaning (some debug’s crap) and fixing structure also adding ReadMe.txt

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 16 April 2011 11:30 AM   [ # 6 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

[UPDATED]V 1.0.3

MORE FEATURES(All old feature still there):

1. Get Crawled Web Content.
2. Optimize maintained rendered html : css and js path.

// Wanna see how CodeIgniter Forums looks like in searching engine's spider eyes ?
$this->load->library('proxy');
$this->proxy->crawl('ellislab.com/forums',TRUE); 

Download Proxy Library on wiki

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 17 April 2011 02:19 PM   [ # 7 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

No one intend to play with this one yet?

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 21 April 2011 11:58 PM   [ # 8 ]   [ Rating: 0 ]
Joined: 2010-11-27
30 posts

Hi i have some question about this lib :

1. After i crawl and get webpage return with this, how i can get special information from this page?

Before i have used HTML Dom Parse to get info from site.

2. Anything different from browser by this lib and real user browser

 
Posted: 22 April 2011 01:27 PM   [ # 9 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts
tieungao - 22 April 2011 03:58 AM

1. After i crawl and get webpage return with this, how i can get special information from this page?

Before i have used HTML Dom Parse to get info from site.

crawl() function will generates an informative array coressponding with url. Including meta tags and anchor(link). If you want to get full response, use site() or head(). Its have (optional) render functionality, which maintain image, anchor/link, js and css.

tieungao - 22 April 2011 03:58 AM

2. Anything different from browser by this lib and real user browser

Basicly, It simulates a real web browser, only that you use it with lines of code rather than mouse and keyboard.

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 22 April 2011 05:23 PM   [ # 10 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

[UPDATED]V 1.0.4

MORE FEATURES(All old feature still there):

Google Geocoding API Functionality.

Geocoding is the process of converting addresses (like “1600 Amphitheatre Parkway, Mountain View, CA”) into geographic coordinates (like latitude 37.423021 and longitude -122.083739), which you can use to place markers or position the map. The Google Geocoding API provides a direct way to access a geocoder via an HTTP request. Additionally, the service allows you to perform the converse operation (turning coordinates into addresses); this process is known as “reverse geocoding.”

$this->load->library('proxy');
// Generate json response by address
$map_data $this->proxy->geocode('1600 Amphitheatre Parkway, Mountain View, CA');
// Generate json response by latlng
$map_data $this->proxy->geocode(array('40.714224','-73.961452'));
// Generate xml response by latlng and set sensor to TRUE
$map_data $this->proxy->geocode(array(40.714224,-73.961452),'XML',TRUE); 

Download Proxy Library on wiki

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 29 April 2011 03:21 PM   [ # 11 ]   [ Rating: 0 ]
Joined: 2011-04-29
1 posts

Great work, this looks really good!

I have been playing with it and I got an error when I tried render a controller/method that I set up:

$this->load->library(‘proxy’);
$this->proxy->controller(‘test/testing/1’,TRUE);

“An Error Was Encountered
php_network_getaddresses: getaddrinfo failed: Name or service not known [ERROR_CODE] = 0”

Do you have any idea if there is anything that I did wrong? Thank you!

 
Posted: 29 April 2011 11:59 PM   [ # 12 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

Are you set your htaccess properly? How you normally access that controller, how its url looks like?

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 30 April 2011 04:17 PM   [ # 13 ]   [ Rating: 0 ]
Joined: 2010-03-09
222 posts

Hi man!

This lib looks pretty useful, does the function return sorce code of page or it just display the page. I mean can I work with page (I want to make a bot), or can I just show it?

Tom.

 Signature 

I am sorry for my English ... It’s poor but I am still learning it.

 
Posted: 30 April 2011 04:44 PM   [ # 14 ]   [ Rating: 0 ]
Avatar
Joined: 2010-12-20
1591 posts

You can do both!

$this->load->library('proxy');
// You can save the full response in a variable
$some_site $this->proxy->site('www.somesite.com');
// If you want directly display it, add second params to true
$this->proxy->site('www.somesite.com',TRUE); 

Download it on Wiki, and you can see the full reference on Readme.txt

 Signature 

“In Code We Trust.”


CI Library : Gas ORM | Proxy

 
Posted: 30 April 2011 05:04 PM   [ # 15 ]   [ Rating: 0 ]
Joined: 2010-03-09
222 posts

It seems like a really great job! I have bookmarked this page and I will probably use it my next project. I’ll be in June or July, than I’ll write you some feedback.

Thanks anyway, for this great lib.

Tom.

 Signature 

I am sorry for my English ... It’s poor but I am still learning it.

 
1 of 3
1