Google Place Big Data Report
是的, 各位朋友! 麥大叔花了我近兩周的時間, 日夜苦幹, 連遊戲都不打,
終於街景資料庫快要可以上線了!
» Read More
Google Place Big Data Report2016-03-06
StreetView.cc Development Diary2016-03-15
Streetview.cc Development Diary
Kind of a note to memorize something that I may forget in few days after....
Ver 2.1 Released:
Public
StreetView: Display places with Google See-Inside features.
Coverage: Reveal areas which have been scanned by system.
SeeInside: SeeInside a place with the UI combines Google Maps(pegman) and its StreetView image on the same page.
Private
SECRETS
Website Version
2016/10/07 - Ver 2.1 Released. all major function revised.
2016S2S3 - Development https://Place.Report website
2016/4/13 - Revise See-Inside judgement programming (v1.1)
2016/4/12 - Comment_Service implemented
2016/4/01 - Ver 1.3 Released. mysqli implemented
2016/3/25 - Ver 1.2 Released. New function: MapScan, Providing Linear and area two types scanning.
2016/3/22 - Website Automation Test Success!!
2016/3/17 - Improve processing efficiency by using curl instead of file_get_contents
2016/3/15 - Big-Data Place Report trial. (providing Place Ranks)
2016/3/12 - Ver 1.1 Released. New functions: Coverage, PlaceRank, SeeInside
2016/3/12 - Integrated g.ymc2.com, an URL Shorten API. a JSON application
2016/3/10 - Ver 1.0 released to DOMAIN: streetview.cc HOST: netfirms.com
Function: Home, OnePageReport, PlaceSearch, MapProcess
2016/3/01 Defined three modules, placeid, cid and cbk, to figure out if a store has the SEE-INSIDE feature or not.
2016/2/15 Start Coding .........A Google Maps Javascript V3 Journey....
--
Problems/Issues/Improvements Ideas 2.1:
1. CSS Layout for mobile device. TBH, i don't really care about this for sure.
2. Google API limitation. a Major concerns
3. Facebook Graph API
Update Place.DB (Automatic and Manual)
Define a mid-term relationship for Place.DB and Seeing.DB (maybe a Pano.DB)?
Useful Statistic Data
Replace mysql to my_sqli sentence
Rethink the function: judge_seeinside, judgement see-inside...
Implement google analytic
IP rule for certain function
--
Major Fixed:
1. How to process place results generated by Google Maps API (Async) to db?
-> 1) generate a click button within the async function.
-> 2) generate an auto-run PHP page.
2. How to curl the place results generated by Google Maps API (async) to a html.element?
-> 1) Curl does not work on async page. (empty element)
-> 2) Include the Google Maps API script on the parent page
3. Improve server efficiency:
-> A test reveals that curl is 50 percent faster than file_get_contents.
-> Replace file_get_contents with curl function.
4. Website automation test solution:
-> Tried Selenium webdriver, but was unable to figure out a way to make it work on wamp server.
-> Tried Selenium IDE, Works, however it had file size limits for processing codes(20KB?), and also there was no flow control (if-else) at the moment for testing purpose.
-> Turned out a solution which is to connect the server with localhost, and write PHP pages to the server via fopen(ftp), and then automatically run these pages to process results.
5. Encode the imgurl of all pano:
-> Applied JSON to communicate with g.ymc2.com (url shorten service)
6. fopen($ftp_path): failed to open stream at certain time ?
-> It seems like the hosting server issue (connection error or what?)
-> Use curl to do ftp method. (not sure if doing this is right, but it temporally works!)
7. The owner-response-rate (Google Reviews)
-> Find the recent 10 reviews and check to see if response from the owner exists
PHP curl vs file_get_contents2016-03-17
a comparison of time-consuming for curl() and file_get_contents()
1. Target: Google generated page
2. Programming: Basically string processing, apply preg_match, preg_replace, preg_split after retrieving HTML content.
3. RESULT
--
Result 1
$curl_info[size_download]: 542711
curl get contents timing => 0.5671, 0.5821, 0.7801
file_get_contents timing => 0.8851, 0.8641, 0.9081
--
Result 2
$curl_info[size_download]: 554880
curl get contents timing => 0.7771, 0.5741, 0.6331
file_get_contents timing => 0.9991, 1.2871, 0.9111
--
Result 3
$curl_info[size_download]: 561892
curl get contents timing => 0.7111, 0.7091, 0.6931
file_get_contents timing => 1.0921, 1.3880, 0.9640
--
Conclusion
Curl is 50percent faster than file_get_contents when getting a 550kb file size.
1. Target: Google generated page
2. Programming: Basically string processing, apply preg_match, preg_replace, preg_split after retrieving HTML content.
3. RESULT
--
Result 1
$curl_info[size_download]: 542711
curl get contents timing => 0.5671, 0.5821, 0.7801
file_get_contents timing => 0.8851, 0.8641, 0.9081
--
Result 2
$curl_info[size_download]: 554880
curl get contents timing => 0.7771, 0.5741, 0.6331
file_get_contents timing => 0.9991, 1.2871, 0.9111
--
Result 3
$curl_info[size_download]: 561892
curl get contents timing => 0.7111, 0.7091, 0.6931
file_get_contents timing => 1.0921, 1.3880, 0.9640
--
Conclusion
Curl is 50percent faster than file_get_contents when getting a 550kb file size.
mysql 改 mysqli 語法2016-03-30
自虐啊!!! 改什麼mysqli, 瘋了嗎?
寫mysql寫的好好的, 改個屁 mysqli!
好了, 以上發洩完畢, 為了及早準備PHP7.0的時代,
麥大叔這兩天花了點時間改了note資料庫的程式碼 (接下來還有幾個網站要改...哭哭)
主要異動是把即將缷任的mysql語句改為mysqli,
過程其實還好, 只要確定原本的輸出是什麼, mysqli 也能輸出同樣的東西即可。
比較麻煩的是取出多筆資料時要做的迴圈, 會需要一些改變。
心得
1. mysqli 語法改用物件的方式寫會比較順利
$link = @mysql_connect(DB_HOST,DB_USER,DB_PASSWORD);
$db = new mysqli(DB_HOST,DB_USER,DB_PASSWORD,DB_NAME);
2. mysqli 多筆資料時的輸出情況, 以fetch_object做廻圈, 再用json去轉陣列
3. SQL語法包括UPDATE, INSERT INTO時, 宣告字碼 utf8, 避免存入亂碼
$db->set_charset("utf8");
以上, 供後人參考!
寫mysql寫的好好的, 改個屁 mysqli!
好了, 以上發洩完畢, 為了及早準備PHP7.0的時代,
麥大叔這兩天花了點時間改了note資料庫的程式碼 (接下來還有幾個網站要改...哭哭)
主要異動是把即將缷任的mysql語句改為mysqli,
過程其實還好, 只要確定原本的輸出是什麼, mysqli 也能輸出同樣的東西即可。
比較麻煩的是取出多筆資料時要做的迴圈, 會需要一些改變。
心得
1. mysqli 語法改用物件的方式寫會比較順利
$link = @mysql_connect(DB_HOST,DB_USER,DB_PASSWORD);
$db = new mysqli(DB_HOST,DB_USER,DB_PASSWORD,DB_NAME);
2. mysqli 多筆資料時的輸出情況, 以fetch_object做廻圈, 再用json去轉陣列
$result = $db->query($sql);
if($result){
while ($row = $result->fetch_object()){ // Cycle through results
$stdClass[] = $row;
}
$result->close(); // Free result set
}
$val = json_decode(json_encode($stdClass), true); //Transform stdClass to Array
$val = array_map('array_values', $val); //remove Key of the Array (may not required)
if($result){
while ($row = $result->fetch_object()){ // Cycle through results
$stdClass[] = $row;
}
$result->close(); // Free result set
}
$val = json_decode(json_encode($stdClass), true); //Transform stdClass to Array
$val = array_map('array_values', $val); //remove Key of the Array (may not required)
3. SQL語法包括UPDATE, INSERT INTO時, 宣告字碼 utf8, 避免存入亂碼
$db->set_charset("utf8");
以上, 供後人參考!
留言功能再開2016-04-12
留言功能再開
自從把 wordpress 關了之後, 留言功能就死掉了!!
wp_comments的資料庫放了很久, 一直沒有動力去使用它~ (笑)
不過這兩個月由於刻 streetview.cc 的關係,
PHP功力大大增進, 猶如一顆大補丸啊!!
於是就把留言的功能做起來, 時間還蠻恰好的, 正好關了將近一年的時間!
各位朋友! 麥大叔留言板再開!
功能應該會逐漸加回來, 目前通通預設驚聲尖叫當做小圖。 (我超愛這個表情的啦~)
留言服務開發記錄2016-04-15
留言服務開發記錄 Comment Service Development
若不意外, 一般網站留言的功能都是寫在自有的資料庫中,
不過若跨站別有太多網店需要處理得情況時, 寫起來會有點白白的啊, 一直 copy+paste 有點搞笑哦!
所以, 這次麥大叔用一個主資料庫處理所有留言。再用 json 在不同站別做資料傳遞。
在應用上只要在其它各站呼叫函式就自動生成留言了, 管理上也比較好管理哦!
目前公開的版本是 v1.0, 主要實現了留言功能 (這不廢話嗎?)
預期待開發/解決的問題如下:
1. 機器人踹開:
現在我還在觀察機器人的運作的方式, 若都是 curl 留言機器人,
就直接 ajax 把 sumbit button 處理欄位就好啦!
爬都爬不到; 除非有大師用 solenium 之類的來寫, 那也沒辦法了。
我是不太想用什麼文字啊影像之類的驗證碼, 大大降低 User Experience 的分數
2. Mobile Friendly 行動裝置最佳化
說實話, 每次寫PHP網頁, 我最花時間的就是試這些 css 的東西, 比寫code還痛苦!
所以, 這mobile friendly 永遠都是問題, 一直在我心裡隱隱作痛...
3. FB-account Login / Google-account Login
還要加個用FB/Google一鍵登入留言的機制。
這個理論不難, 一個api的轉換而已...
目前大致先降。 有想到其它後補
©Max Chen 2003-2024 V2.1