Python: Script A Google Autosuggest Extract of Trends for Your Niche Search Keywords

Script Python Biex Tiġbor Xejriet Autosuggest

Kulħadd jħobb Google Trends, iżda huwa daqsxejn delikat meta niġu għal Long Tail Keywords. Aħna lkoll nħobbu l-uffiċjal servizz ta 'tendenzi google biex ikollok għarfien dwar l-imġieba tat-tfittxija. Madankollu, żewġ affarijiet jipprevjenu lil bosta milli jużawha għal xogħol solidu;

  1. Meta jkollok bżonn issib keywords ġodda niċċa, hemm mhix biżżejjed dejta fuq Google Trends 
  2. Nuqqas ta 'API uffiċjali biex tagħmel talbiet lil google trends: Meta nagħmlu użu minn moduli bħal pirtrendi, allura rridu nużaw proxy servers, jew inkunu mblukkati. 

F'dan l-artikolu, ser naqsam Skritt Python li ktibna biex nesportaw kliem ewlieni trending permezz ta 'Google Autosuggest.

Iġbed u Aħżen Riżultati Autosuggest Matul iż-Żmien 

Ejja ngħidu li għandna 1,000 kliem ewlieni taż-Żerriegħa biex jintbagħtu lil Google Autosuggest. Bi tpattija, probabbilment ikollna madwar 200,000 longtail keywords. Imbagħad, għandna nagħmlu l-istess ġimgħa wara u nqabblu dawn is-settijiet ta 'dejta biex inwieġbu żewġ mistoqsijiet:

  • Liema mistoqsijiet huma kliem ewlieni ġdid meta mqabbel mal-aħħar darba? Dan huwa probabbilment il-każ li għandna bżonn. Google jaħseb li dawk il-mistoqsijiet qed isiru aktar sinifikanti - billi nagħmlu dan, nistgħu noħolqu s-soluzzjoni tagħna stess Google Autosuggest! 
  • Liema mistoqsijiet huma keywords m'għadhomx trending?

L-iskritt huwa pjuttost faċli, u ħafna mill-kodiċi qsamt hawn. Il-kodiċi aġġornat jiffranka d-dejta minn ġirjiet tal-passat u jqabbel is-suġġerimenti maż-żmien. Evitaw databases ibbażati fuq fajls bħal SQLite biex tagħmilha sempliċi - allura l-ħażna tad-dejta kollha qed tuża fajls CSV hawn taħt. Dan jippermettilek timporta l-fajl f'Excel u tesplora xejriet ta 'kliem ewlieni niċċa għan-negozju tiegħek.

Biex Tutilizza Din l-Iskrittura Python

  1. Daħħal is-sett ta 'kliem ewlieni taż-żerriegħa tiegħek li għandu jintbagħat lill-awtokompletar: keywords.csv
  2. Aġġusta s-settings tal-Iskrittura għall-bżonn tiegħek:
    • LINGWA: default "en"
    • PAJJIŻ: default "us"
  3. Skeda l-iskritt biex jaħdem darba fil-ġimgħa. Tista 'wkoll tmexxih manwalment kif tixtieq.
  4. Uża keyword_suggestions.csv għal aktar analiżi:
    • first_seed: din hija d-data fejn il-mistoqsija dehret għall-ewwel darba fl-autosuggest
    • l-aħħar_ar: id-data fejn il-mistoqsija dehret għall-aħħar darba
    • huwa_ġdid: jekk first_seen == last_seen nissettjaw dan għal Veru - Iffiltra biss fuq dan il-valur biex tikseb it-tfittxijiet ġodda ta 'tendenza fl-awtosuggest ta' Google.

Hawn il-Kodiċi Python

# Pemavor.com Autocomplete Trends
# Author: Stefan Neefischer (stefan.neefischer@gmail.com)
import concurrent.futures
from datetime import date
from datetime import datetime
import pandas as pd
import itertools
import requests
import string
import json
import time

charList = " " + string.ascii_lowercase + string.digits

def makeGoogleRequest(query):
    # If you make requests too quickly, you may be blocked by google 
    time.sleep(WAIT_TIME)
    URL="http://suggestqueries.google.com/complete/search"
    PARAMS = {"client":"opera",
            "hl":LANGUAGE,
            "q":query,
            "gl":COUNTRY}
    response = requests.get(URL, params=PARAMS)
    if response.status_code == 200:
        try:
            suggestedSearches = json.loads(response.content.decode('utf-8'))[1]
        except:
            suggestedSearches = json.loads(response.content.decode('latin-1'))[1]
        return suggestedSearches
    else:
        return "ERR"

def getGoogleSuggests(keyword):
    # err_count1 = 0
    queryList = [keyword + " " + char for char in charList]
    suggestions = []
    for query in queryList:
        suggestion = makeGoogleRequest(query)
        if suggestion != 'ERR':
            suggestions.append(suggestion)

    # Remove empty suggestions
    suggestions = set(itertools.chain(*suggestions))
    if "" in suggestions:
        suggestions.remove("")
    return suggestions

def autocomplete(csv_fileName):
    dateTimeObj = datetime.now().date()
    #read your csv file that contain keywords that you want to send to google autocomplete
    df = pd.read_csv(csv_fileName)
    keywords = df.iloc[:,0].tolist()
    resultList = []

    with concurrent.futures.ThreadPoolExecutor(max_workers=MAX_WORKERS) as executor:
        futuresGoogle = {executor.submit(getGoogleSuggests, keyword): keyword for keyword in keywords}

        for future in concurrent.futures.as_completed(futuresGoogle):
            key = futuresGoogle[future]
            for suggestion in future.result():
                resultList.append([key, suggestion])

    # Convert the results to a dataframe
    suggestion_new = pd.DataFrame(resultList, columns=['Keyword','Suggestion'])
    del resultList

    #if we have old results read them
    try:
        suggestion_df=pd.read_csv("keyword_suggestions.csv")
        
    except:
        suggestion_df=pd.DataFrame(columns=['first_seen','last_seen','Keyword','Suggestion'])
    
    suggestionCommon_list=[]
    suggestionNew_list=[]
    for keyword in suggestion_new["Keyword"].unique():
        new_df=suggestion_new[suggestion_new["Keyword"]==keyword]
        old_df=suggestion_df[suggestion_df["Keyword"]==keyword]
        newSuggestion=set(new_df["Suggestion"].to_list())
        oldSuggestion=set(old_df["Suggestion"].to_list())
        commonSuggestion=list(newSuggestion & oldSuggestion)
        new_Suggestion=list(newSuggestion - oldSuggestion)
         
        for suggest in commonSuggestion:
            suggestionCommon_list.append([dateTimeObj,keyword,suggest])
        for suggest in new_Suggestion:
            suggestionNew_list.append([dateTimeObj,dateTimeObj,keyword,suggest])
    
    #new keywords
    newSuggestion_df = pd.DataFrame(suggestionNew_list, columns=['first_seen','last_seen','Keyword','Suggestion'])
    #shared keywords with date update
    commonSuggestion_df = pd.DataFrame(suggestionCommon_list, columns=['last_seen','Keyword','Suggestion'])
    merge=pd.merge(suggestion_df, commonSuggestion_df, left_on=["Suggestion"], right_on=["Suggestion"], how='left')
    merge = merge.rename(columns={'last_seen_y': 'last_seen',"Keyword_x":"Keyword"})
    merge["last_seen"].fillna(merge["last_seen_x"], inplace=True)
    del merge["last_seen_x"]
    del merge["Keyword_y"]
    
    #merge old results with new results
    frames = [merge, newSuggestion_df]
    keywords_df =  pd.concat(frames, ignore_index=True, sort=False)
    # Save dataframe as a CSV file
    keywords_df['first_seen'] = pd.to_datetime(keywords_df['first_seen'])
    keywords_df = keywords_df.sort_values(by=['first_seen','Keyword'], ascending=[False,False])   
    keywords_df['first_seen']= pd.to_datetime(keywords_df['first_seen'])
    keywords_df['last_seen']= pd.to_datetime(keywords_df['last_seen'])
    keywords_df['is_new'] = (keywords_df['first_seen']== keywords_df['last_seen'])
    keywords_df=keywords_df[['first_seen','last_seen','Keyword','Suggestion','is_new']]
    keywords_df.to_csv('keyword_suggestions.csv', index=False)

# If you use more than 50 seed keywords you should slow down your requests - otherwise google is blocking the script
# If you have thousands of seed keywords use e.g. WAIT_TIME = 1 and MAX_WORKERS = 5
WAIT_TIME = 0.2
MAX_WORKERS = 20
# set the autocomplete language
LANGUAGE = "en"
# set the autocomplete country code - DE, US, TR, GR, etc..
COUNTRY="US"
# Keyword_seed csv file name. One column csv file.
#csv_fileName="keyword_seeds.csv"
CSV_FILE_NAME="keywords.csv"
autocomplete(CSV_FILE_NAME)
#The result will save in keyword_suggestions.csv csv file

Niżżel l-Iskrittura Python

What do you think?

Dan is-sit juża Akismet biex inaqqas l-ispam. Tgħallem kif il-kumment tiegħek huwa pproċessat.