1. The brief

Imagine working for a digital marketing agency, and the agency is approached by a massive online retailer of furniture. They want to test our skills at creating large campaigns for all of their website. We are tasked with creating a prototype set of keywords for search campaigns for their sofas section. The client says that they want us to generate keywords for the following products:

  • sofas
  • convertible sofas
  • love seats
  • recliners
  • sofa beds

The brief: The client is generally a low-cost retailer, offering many promotions and discounts. We will need to focus on such keywords. We will also need to move away from luxury keywords and topics, as we are targeting price-sensitive customers. Because we are going to be tight on budget, it would be good to focus on a tightly targeted set of keywords and make sure they are all set to exact and phrase match.

Based on the brief above we will first need to generate a list of words, that together with the products given above would make for good keywords. Here are some examples:

  • Products: sofas, recliners
  • Words: buy, prices

The resulting keywords: 'buy sofas', 'sofas buy', 'buy recliners', 'recliners buy', 'prices sofas', 'sofas prices', 'prices recliners', 'recliners prices'.

As a final result, we want to have a DataFrame that looks like this:

Campaign Ad Group Keyword Criterion Type
Campaign1 AdGroup_1 keyword 1a Exact
Campaign1 AdGroup_1 keyword 1a Phrase
Campaign1 AdGroup_1 keyword 1b Exact
Campaign1 AdGroup_1 keyword 1b Phrase
Campaign1 AdGroup_2 keyword 2a Exact
Campaign1 AdGroup_2 keyword 2a Phrase

The first step is to come up with a list of words that users might use to express their desire in buying low-cost sofas.

words = ['buy', 'promotion', 'discount', 'price', 'promo', 'shop']

# Print list of words
# ... YOUR CODE FOR TASK 1 ...
print(words)
['buy', 'promotion', 'discount', 'price', 'promo', 'shop']

2. Combine the words with the product names

Imagining all the possible combinations of keywords can be stressful! But not for us, because we are keyword ninjas! We know how to translate campaign briefs into Python data structures and can imagine the resulting DataFrames that we need to create.

Now that we have brainstormed the words that work well with the brief that we received, it is now time to combine them with the product names to generate meaningful search keywords. We want to combine every word with every product once before, and once after, as seen in the example above.

As a quick reminder, for the product 'recliners' and the words 'buy' and 'price' for example, we would want to generate the following combinations:

buy recliners
recliners buy
price recliners
recliners price

and so on for all the words and products that we have.

products = ['sofas', 'convertible sofas', 'love seats', 'recliners', 'sofa beds']

# Create an empty list
keywords_list = list()

# Loop through products
for product in products:
    # Loop through words
    for word in words:
        # Append combinations
        keywords_list.append([product, product + ' ' + word])
        keywords_list.append([product, word + ' ' + product])
        
# Inspect keyword list
from pprint import pprint
pprint(keywords_list)
[['sofas', 'sofas buy'],
 ['sofas', 'buy sofas'],
 ['sofas', 'sofas promotion'],
 ['sofas', 'promotion sofas'],
 ['sofas', 'sofas discount'],
 ['sofas', 'discount sofas'],
 ['sofas', 'sofas price'],
 ['sofas', 'price sofas'],
 ['sofas', 'sofas promo'],
 ['sofas', 'promo sofas'],
 ['sofas', 'sofas shop'],
 ['sofas', 'shop sofas'],
 ['convertible sofas', 'convertible sofas buy'],
 ['convertible sofas', 'buy convertible sofas'],
 ['convertible sofas', 'convertible sofas promotion'],
 ['convertible sofas', 'promotion convertible sofas'],
 ['convertible sofas', 'convertible sofas discount'],
 ['convertible sofas', 'discount convertible sofas'],
 ['convertible sofas', 'convertible sofas price'],
 ['convertible sofas', 'price convertible sofas'],
 ['convertible sofas', 'convertible sofas promo'],
 ['convertible sofas', 'promo convertible sofas'],
 ['convertible sofas', 'convertible sofas shop'],
 ['convertible sofas', 'shop convertible sofas'],
 ['love seats', 'love seats buy'],
 ['love seats', 'buy love seats'],
 ['love seats', 'love seats promotion'],
 ['love seats', 'promotion love seats'],
 ['love seats', 'love seats discount'],
 ['love seats', 'discount love seats'],
 ['love seats', 'love seats price'],
 ['love seats', 'price love seats'],
 ['love seats', 'love seats promo'],
 ['love seats', 'promo love seats'],
 ['love seats', 'love seats shop'],
 ['love seats', 'shop love seats'],
 ['recliners', 'recliners buy'],
 ['recliners', 'buy recliners'],
 ['recliners', 'recliners promotion'],
 ['recliners', 'promotion recliners'],
 ['recliners', 'recliners discount'],
 ['recliners', 'discount recliners'],
 ['recliners', 'recliners price'],
 ['recliners', 'price recliners'],
 ['recliners', 'recliners promo'],
 ['recliners', 'promo recliners'],
 ['recliners', 'recliners shop'],
 ['recliners', 'shop recliners'],
 ['sofa beds', 'sofa beds buy'],
 ['sofa beds', 'buy sofa beds'],
 ['sofa beds', 'sofa beds promotion'],
 ['sofa beds', 'promotion sofa beds'],
 ['sofa beds', 'sofa beds discount'],
 ['sofa beds', 'discount sofa beds'],
 ['sofa beds', 'sofa beds price'],
 ['sofa beds', 'price sofa beds'],
 ['sofa beds', 'sofa beds promo'],
 ['sofa beds', 'promo sofa beds'],
 ['sofa beds', 'sofa beds shop'],
 ['sofa beds', 'shop sofa beds']]

3. Convert the list of lists into a DataFrame

Now we want to convert this list of lists into a DataFrame so we can easily manipulate it and manage the final output.

# ... YOUR CODE FOR TASK 3 ...
import pandas as pd

# Create a DataFrame from list
keywords_df = pd.DataFrame.from_records(keywords_list)

# Print the keywords DataFrame to explore it
# ... YOUR CODE FOR TASK 3 ...
keywords_df.head()
0 1
0 sofas sofas buy
1 sofas buy sofas
2 sofas sofas promotion
3 sofas promotion sofas
4 sofas sofas discount

4. Rename the columns of the DataFrame

Before we can upload this table of keywords, we will need to give the columns meaningful names. If we inspect the DataFrame we just created above, we can see that the columns are currently named 0 and 1. Ad Group (example: "sofas") and Keyword (example: "sofas buy") are much more appropriate names.

keywords_df.rename(columns=[0, 1])
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-57-c32ddb7ee77c> in <module>()
      1 # Rename the columns of the DataFrame
----> 2 keywords_df.rename(columns=[0, 1])

/usr/local/lib/python3.5/dist-packages/pandas/util/_decorators.py in wrapper(*args, **kwargs)
    185         @wraps(func)
    186         def wrapper(*args, **kwargs):
--> 187             return func(*args, **kwargs)
    188 
    189         if not PY2:

/usr/local/lib/python3.5/dist-packages/pandas/core/frame.py in rename(self, *args, **kwargs)
   3779         kwargs.pop('axis', None)
   3780         kwargs.pop('mapper', None)
-> 3781         return super(DataFrame, self).rename(**kwargs)
   3782 
   3783     @Substitution(**_shared_doc_kwargs)

/usr/local/lib/python3.5/dist-packages/pandas/core/generic.py in rename(self, *args, **kwargs)
    972                 level = self.axes[axis]._get_level_number(level)
    973             result._data = result._data.rename_axis(f, axis=baxis, copy=copy,
--> 974                                                     level=level)
    975             result._clear_item_cache()
    976 

/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py in rename_axis(self, mapper, axis, copy, level)
   3338         """
   3339         obj = self.copy(deep=copy)
-> 3340         obj.set_axis(axis, _transform_index(self.axes[axis], mapper, level))
   3341         return obj
   3342 

/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py in _transform_index(index, func, level)
   5296         return MultiIndex.from_tuples(items, names=index.names)
   5297     else:
-> 5298         items = [func(x) for x in index]
   5299         return Index(items, name=index.name, tupleize_cols=False)
   5300 

/usr/local/lib/python3.5/dist-packages/pandas/core/internals.py in <listcomp>(.0)
   5296         return MultiIndex.from_tuples(items, names=index.names)
   5297     else:
-> 5298         items = [func(x) for x in index]
   5299         return Index(items, name=index.name, tupleize_cols=False)
   5300 

TypeError: 'list' object is not callable

5. Add a campaign column

Now we need to add some additional information to our DataFrame. We need a new column called Campaign for the campaign name. We want campaign names to be descriptive of our group of keywords and products, so let's call this campaign 'SEM_Sofas'.

# ... YOUR CODE FOR TASK 5 ...
keywords_df['Campaign'] = 'SEM_Sofas'

6. Create the match type column

There are different keyword match types. One is exact match, which is for matching the exact term or are close variations of that exact term. Another match type is broad match, which means ads may show on searches that include misspellings, synonyms, related searches, and other relevant variations.

Straight from Google's AdWords documentation:

In general, the broader the match type, the more traffic potential that keyword will have, since your ads may be triggered more often. Conversely, a narrower match type means that your ads may show less often—but when they do, they’re likely to be more related to someone’s search.

Since the client is tight on budget, we want to make sure all the keywords are in exact match at the beginning.

# ... YOUR CODE FOR TASK 6 ...
keywords_df['Criterion Type'] = 'Exact'

7. Duplicate all the keywords into 'phrase' match

The great thing about exact match is that it is very specific, and we can control the process very well. The tradeoff, however, is that:

  1. The search volume for exact match is lower than other match types
  2. We can't possibly think of all the ways in which people search, and so, we are probably missing out on some high-quality keywords.

So it's good to use another match called phrase match as a discovery mechanism to allow our ads to be triggered by keywords that include our exact match keywords, together with anything before (or after) them.

Later on, when we launch the campaign, we can explore with modified broad match, broad match, and negative match types, for better visibility and control of our campaigns.

keywords_phrase = keywords_df.copy()

# Change criterion type match to phrase
# ... YOUR CODE FOR TASK 7 ...
keywords_phrase['Criterion Type'] = 'Phrase'

# Append the DataFrames
keywords_df_final = keywords_df.append(keywords_phrase)

8. Save and summarize!

To upload our campaign, we need to save it as a CSV file. Then we will be able to import it to AdWords editor or BingAds editor. There is also the option of pasting the data into the editor if we want, but having easy access to the saved data is great so let's save to a CSV file!

Looking at a summary of our campaign structure is good now that we've wrapped up our keyword work. We can do that by grouping by ad group and criterion type and counting by keyword. This summary shows us that we assigned specific keywords to specific ad groups, which are each part of a campaign. In essence, we are telling Google (or Bing, etc.) that we want any of the words in each ad group to trigger one of the ads in the same ad group. Separately, we will have to create another table for ads, which is a task for another day and would look something like this:

Campaign Ad Group Headline 1 Headline 2 Description Final URL
SEM_Sofas Sofas Looking for Quality Sofas? Explore Our Massive Collection 30-day Returns With Free Delivery Within the US. Start Shopping Now DataCampSofas.com/sofas
SEM_Sofas Sofas Looking for Affordable Sofas? Check Out Our Weekly Offers 30-day Returns With Free Delivery Within the US. Start Shopping Now DataCampSofas.com/sofas
SEM_Sofas Recliners Looking for Quality Recliners? Explore Our Massive Collection 30-day Returns With Free Delivery Within the US. Start Shopping Now DataCampSofas.com/recliners
SEM_Sofas Recliners Need Affordable Recliners? Check Out Our Weekly Offers 30-day Returns With Free Delivery Within the US. Start Shopping Now DataCampSofas.com/recliners

Together, these tables get us the sample keywords -> ads -> landing pages mapping shown in the diagram below.

Keywords-Ads-Landing pages flow

# ... YOUR CODE FOR TASK 8 ...
keywords_df_final.to_csv('keywords.csv', index=False)

# View a summary of our campaign work
summary = keywords_df_final.groupby(['Ad Group', 'Criterion Type'])['Keyword'].count()
print(summary)
---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
<ipython-input-64-27d087b4c3d3> in <module>()
      4 
      5 # View a summary of our campaign work
----> 6 summary = keywords_df_final.groupby(['Ad Group', 'Criterion Type'])['Keyword'].count()
      7 print(summary)

/usr/local/lib/python3.5/dist-packages/pandas/core/generic.py in groupby(self, by, axis, level, as_index, sort, group_keys, squeeze, observed, **kwargs)
   6663         return groupby(self, by=by, axis=axis, level=level, as_index=as_index,
   6664                        sort=sort, group_keys=group_keys, squeeze=squeeze,
-> 6665                        observed=observed, **kwargs)
   6666 
   6667     def asfreq(self, freq, method=None, how=None, normalize=False,

/usr/local/lib/python3.5/dist-packages/pandas/core/groupby/groupby.py in groupby(obj, by, **kwds)
   2150         raise TypeError('invalid type: %s' % type(obj))
   2151 
-> 2152     return klass(obj, by, **kwds)
   2153 
   2154 

/usr/local/lib/python3.5/dist-packages/pandas/core/groupby/groupby.py in __init__(self, obj, keys, axis, level, grouper, exclusions, selection, as_index, sort, group_keys, squeeze, observed, **kwargs)
    597                                                     sort=sort,
    598                                                     observed=observed,
--> 599                                                     mutated=self.mutated)
    600 
    601         self.obj = obj

/usr/local/lib/python3.5/dist-packages/pandas/core/groupby/groupby.py in _get_grouper(obj, key, axis, level, sort, observed, mutated, validate)
   3289                 in_axis, name, level, gpr = False, None, gpr, None
   3290             else:
-> 3291                 raise KeyError(gpr)
   3292         elif isinstance(gpr, Grouper) and gpr.key is not None:
   3293             # Add key to exclusions

KeyError: 'Ad Group'