Multiview Deep Learning for Predicting Twitter Users' Location

Introduction

Over the last ten years, social networks have grown and engaged a massive amount of users. Among them, Twitter is one of the most popular, reaching over 300 million users by the $4^{th}$ quarter of $2017$ . On Twitter, users publish short messages of 140 characters or less called tweets, which can be seen by followers or by the public. Tweets can also be re-published by users who have seen the tweets, a process known as retweeting. This way, information can be spread quickly and widely throughout the whole Twitter network. Twitter can even be considered as a human-powered sensing network, with a lot of useful information, yet, in an unstructured form. For this reason, automatic mining and extracting meaningful information from the massive amount of Twitter data is of great significance , .

A very useful piece of information on Twitter is user location, which enables several applications including event detection , online community analysis , social unrest forecasting and location-based recommendation . As another example, user location information can be useful for online marketers and governments to understand trends and patterns ranging from customer and citizen feedback to the mapping of epidemics in concerned geographical areas . In 2009, Twitter enabled a geo-tagging feature, with which users can choose to geo-tag their tweets while posting. However, the majority of tweets are not geo-tagged by the users . Alternatively, users’ location might be available via their profile data. Nonetheless, not many users disclose their location via their Twitter profile, or the provided information is often unreliable. For example, a user might share vague or non-existent places such as ”Everywhere” and ”Small town, RW Texas”. This results in a quest for geolocation algorithms that can automatically analyze and infer the location of Twitter users.

The Twitter geolocation problem can be addressed at two different levels, namely, the tweet level and the user level. The former aims at predicting the location of single tweets, while the latter aims at inferring the location of a user from the data generated by that user. The geolocation of single tweets is extremely difficult due to the limited availability of information. Research on single tweet geolocation has been conducted , but a good accuracy can be achieved only under specific constraints, which are normally not applicable in real-life situations. On the other hand, the Twitter geolocation at user level, also refered to as Twitter user geolocation, is more common, with plenty of methods described in the literature . In this paper, we focus on the geolocation problem at user level instead of tweet level.

The Twitter user geolocation problem can be formulated under a classification or a regression setting. Under the classification setting, one can predict the location of users in terms of geographical regions, such as countries, states and cities. Under the regression setting, the task is to estimate the exact geocoordinates of the users. Both prediction settings are considered in this paper. It is worth mentioning that we address the regression problem from a classification point of view. Towards this end, we employ a map partitioning technique to divide the concerned geographical area into small regions corresponding to classes. The exact geocoordinates of Twitter users can be estimated using the classes’ centroids.

In the Twitter user geolocation literature, most of the existing algorithms follow either a content-based approach or a network-based approach. Content-based methods extract information from the textual contents of tweets to predict user locations . Network-based methods, on the other hand, employ connections between users for geolocation . Both approaches have achieved good geolocation accuracy .

This paper explores a more generic approach, which inherits the advantages of both content-based and network-based strategies. Our approach leverages recent advances in deep neural networks (i.e., deep learning) and multiview learning. Deep neural networks , have been proven to be very effective in many domains including image classification , machine translation , and compressive sensing . On the other hand, multiview learning is an emerging paradigm encompassing methods that learn from examples with multiple representations showing a great progress recently . In Twitter user geolocation, the views can be different types of information available on Twitter such as text and metadata, or even features extracted from the tweets themselves.

Our contributions in this work are as follows:

We propose a generic multiview neural network architecture, named multi-entry neural network (MENET), for Twitter user geolocation. MENET is capable of combining multiview features into a unified model to infer users’ location.

We propose to incorporate four specific types of features to realize MENET for Twitter user geolocation. These features capture the textual information (TF-IDF, doc2vec ), the user interaction network structure (node2vec ), and the time-related user behavior.

We show the effectiveness of using map partitioning techniques in Twitter user geolocation, especially with Google’s S2 partitioning libraryhttps://code.google.com/archive/p/s2-geometry-library/. We have achieved state-of-the-art results on several popular datasets with these partitioning techniques.

We show a thorough analysis on the importance of input features and the impact of partitioning strategies on the performance of MENET.

The remainder of this paper is organized as follows. In Section 2, we review related works. Section 3 describes our method in details, including the model architecture, feature learning, feature extraction and how we improve our model with the density-driven map partitioning technique. Section 4 describes the performance criteria, the pre-processing procedures and details the parameter setting of our method. The results of our experiments are also presented in this section. Finally, we draw the conclusion and discuss future work in Section 5.

Related Work

Most current approaches for predicting the location of Twitter users are based either on user-generated content or on the social ties. The first approach, which has been investigated thoroughly, uses textual features from tweets to build location predictive models. The latter arises from an observation that a user often interacts with people in nearby areas , and exploits the network connections of users. This section will bring a closer look on recently published works for both approaches.

Plenty of content-based methods have been proposed for Twitter user geolocation. Geographical topic models consider tweets and locations as the outputs of a generative process incorporating topics and regions as latent variables, thus geo-locating users by seeking to recover these variables. An alternative approach is using geographical Gaussian Mixture Models (GMMs) to model the distribution of terms of tweets across geographical areas. By calculating a weighted sum of corresponding GMMs on terms of tweets, a geographical density function can be found, revealing the location at the single tweet level. A smilar approach, making use of GMMs, is introduced by Chang et al. , where a GMM model is fit to the conditional probability of a certain city, given a term. Char et al. estimate location by exploiting the expressiveness of sparse coding and the advances in dictionary learning to obtain the state of the art on a benchmark dataset named GeoText . Recently, several methods have addressed the Twitter user geolocation problem using deep learning. For example, Liu and Inkpen train stacked denoising autoencoders for predicting regions, states, and geographical coordinates . These vanilla models obtain quite good results with a pre-training procedure. These methods, however, do not take into account the natural distribution of Twitter users in the considered datasets over the different regions of interest. Concretely, the density of Twitter users is much higher in inner-city areas than countrysides. To exploit this attribute, grid-based geolocation methods are introduced in , where adaptive or uniform grids are created to partition the datasets into geographical cells at different levels. The prediction of geographical coordinates is then converted to a classification problem using the cells as classes, and off-the-shelf classifiers can be applied directly. This strategy is also used in our method but with a different spliting scheme and with a novel model architecture.

Recent works have shown a correlation between the likelihood of friendship of two social network users and the geographical distance between them . Using this correlation, the location of users can be estimated using their friends’ location. This is the key idea behind the network-based approach. By leveraging the social interactions like bi-directional followingTwitter users follow other people to see their latest updates. Bi-directional following means two users follow each other. and bi-directional mentioningTwitter users can mention other people in their tweets by typing @username. Bi-directional mentioning is the two-way interaction which happens when two users have mentioned each other., one can establish graphs of Twitter users where a label propagation algorithm or its variants are used to identify locations of unlabeled users . The network-based approach has several advantages over the content-based counterpart, including language independence. Also, it does not require training, which is a very resource intensive and time-consuming process on big datasets. However, the inherent weakness of this approach is that it cannot propagate labels (locations) to users that are not connected to the graph. As a result, isolated users remain unlabeled.

To address the problem of isolated users in the network-based approach, unified text and network methods are proposed in , which leverage both the discriminative power of textual information and the representativeness of the users’ graph. In particular, the textual information is used to predict labels for disconnected users before running label propagation algorithms. Additionally, the novelty of the works lies in building a densely undirected graph based on the mentioning of users. This makes a significant improvement in the location prediction. Following , models combining text, metadata and user network features have been introduced . These models have to rely on user profile information including user location, user timezone and user UTC offset. These types of information should be considered unvailable in the Twitter user geolocation context. That is the reason why the three benchmark datasets considered in this paper do not provide the Twitter profile information.

Our method does not rely on the Twitter user profile information. It employs a similar graph of Twitter users derived from tweets as in ; however, instead of propagating labels through the graph, our method trains an embedding mapping function to capture the graph’s structure. The graph feature is then integrated with all other features in a neural network architecture. Our architecture is simpler as it does not require designing a specific architecture for each type of feature like in , thus easier and less resource intensive to train.

Multi-entry Neural Network for Twitter User Geolocation

In Twitter user geolocation, we wish to predict the location of a user using textual information and metadata, obtained from a corpus of tweets sent by the user, as well as information extracted from the user’s network. Using this information, we predict either the area (alias, region), where the user most probably resides, or even the location of the user by means of geocoordinates. Our method addresses this problem as a classification problem. Concretely, for each considered dataset, we subdivide Twitter users into discrete geographical regions, which correspond to classes. We define the centroid of a region by the median value of the geocoordinates of all training users in that region. Once a test user is classified to a certain region, we consider the centroid of that region as the predicted geocoordinates.

We propose a generic neural network model to learn from multiple views of data for Twitter user geolocation. We coin the proposed model MENET. The advantage of this model is the capability of exploiting both content-based and network-based features, as well as other available features concurrently. In this work, we realize MENET with different types of features. These features capture not only the tweets’ content, but also the user network structure and time information. It is worth mentioning that except the time information, all other features are extracted from the tweets’ content. Hence, MENET works even in case tweets’ metadata is not available. Integrating all features into MENET results in a powerful method for geolocation. Combining this method with the Google S2 map partitioning technique, we achieve state-of-the-art results in several Twitter user geolocation benchmarks. This section presents our MENET model and the different types of employed features in detail.

Our MENET architecture is illustrated in Fig. 1. The model leverages different features extracted from the tweets’ content and metadata. Each corresponds to one view of the network. In Fig. 1, $k$ features are put into $k$ individual branches. Each branch can contain multiple hidden layers allowing to learn higher order features.

Given multiple views of the input data, a straightforward approach to combine them is to use vector concatenation. Nevertheless, we argue that our architecture is more effective. Simple vector concatenation often does not fully utilize the power of multiple features. In MENET, each view is the input to one network branch, which comprises of a number of fully connected hidden layers. In order to learn a non-linear transformation function for each branch, we employ the ReLU activation function after each hidden layer. The ReLU function is efficient for backpropagation and less prone to the vanishing gradient problem than the tanh and sigmoid activation functions, hence, has been used widely in deep learning literature . The outputs of these branches are concatenated making a combined hidden layer. More fully connected layers can be added after this concatenation layer to gain more nonlinearity (see component Post-combined Hidden Layers in Fig. 1). Again, ReLU is used to activate these layers. At the end, we employ a softmax layer to obtain the output probabilities.

We employ the cross-entropy loss as the objective function. Let $N$ be the number of examples and $m$ be the number of classes, then the cross-entropy loss is defined by:

1.2 Training MENET

The parameters of MENET are fine-tuned using a separated set of examples, namely the development set. During training, the classification accuracy of the model on the development set is continuously monitored. If this metric does not improve for a pre-defined amount of consecutive steps $T_{val}$, the training process is stopped. By using the same mechanism, the learning rate is also annealed when the training proceeds.

1.3 Testing MENET

To predict the location of users from the test set, we use the trained MENET model to classify these users into pre-defined classes (regions). The exact geocoordinates of a user is given by the centroid of the respective region. The performance of the MENET model is measured by either the accuracy in case of regional classification or distance error metrics (see Section 4.2) in case of geographical coordinates prediction.

2 Multiview Features

Figure 1 shows the capability of MENET in exploiting data from multiple sources. In the context of Twitter user geolocation, we realize MENET by leveraging features from textual information (Term Frequency - Inverse Document Frequency , doc2vec ), user interaction network (node2vec ) and metadata (timestamp). These features are all extracted from tweets provided they are available. The rest of this section will describe these features and how they are computed.

The Term Frequency - Inverse Document Frequency (TF-IDF) is a statistical measure used to evaluate how important a term is to a document in a collection or corpus. The importance increases proportionally to the number of times the term appears in the document but is offset by the frequency of the term in the corpus. TF-IDF is composed of two components, presented next.

Term Frequency (TF): It measures how often a term occurs in a document. The simplest choice is the raw frequency of the term in a document

where $f_{t,d}$ is the frequency of term $t$ in the document $d$.

Inverse Document Frequency (IDF): It measures the informative quantity a term brings across documents. Concretely, a common term across multiple documents will be given a low weight while a rare term will have a higher weight. The IDF is defined as

with $D$ denoting the whole set of documents.

2.2 The Context Feature

The context feature is a mapping from a variable length block of text (e.g. sentence, paragraph, or entire document) to a fixed-length continuous valued vector. It provides a numerical representation capturing the context of the document. Originally proposed in , the context feature is also referred to as doc2vec or Distributed Representation of Sentences, and it is an extension of the broadly used word2vec model .

The intuition of doc2vec is that a certain context is more likely to produce some sets of words than other contexts. Doc2vec trains an embedding capable of expressing the relation between the context and the corresponding words. To achieve this goal, it employs a simple neural network architecture consisting of one hidden layer without an activation function. A text window samples some nearby words in a document; some of these words are used as inputs to the network and some as outputs. Moreover, an additional input for the document is added to the network bringing the document’s context. The training process is totally unsupervised. After training, the fixed representaion of the document input will capture the context of the whole document. Two architectures were proposed in to learn a document’s representation, namely, Distributed Bag of Words (PV-DBOW) and Distributed Memory (PV-DM) versions of Paragraph Vector. Athough PV-DBOW is a simpler architecture, it has been claimed that PV-DBOW performs robustly if trained on large datasets . Therefore, we select PV-DBOW model to extract the context feature.

In this paper, we train PV-DBOW models using the tweets from the training sets. Later, we extract the context feature vectors for the training, the development and the test sets. Our implementation is based on gensimhttps://radimrehurek.com/gensim/ .

2.3 The Node2vec Feature

Node2vec employs a sampling method referred to as biased Random Walk , which samples nodes belonging to the neighborhood of node $v$, according to discrete transition probabilities between the current node $v$ and the next node $w$. These probabilities depend on the distance between the previous node $u$ and the next node $w$. Denote by $d_{uw}$ the distance in terms of number of edges from node $u$ to node $w$, if the next node coincides with the previous node, then $d_{uw}=0$. If the next node has a direct connection to the previous node, then $d_{uw}=1$, and if the next node is not connected to the previous node, then $d_{uw}=2$. The transition probabilities are defined as follows :

where the parameters $p$ and $q$ are small positive numbers. The random walk sampling runs on nodes to obtain a list of walks. Later, the node’s embeddings are found from the set of walks using the stochastic gradient descent procedure.

In the context of Twitter user geolocation, each node corresponds to a user, while an edge is the connection between two users. We can define these connections by several criteria depending on the availability of data. For example, we may consider that two users are connected when actions such as following, mentioning or retweeting are detected. In this paper, the content of tweet messages is used to build graph connections. Similar to , , we construct an undirected user graph by employing mention connections. First, we create a unique set $V$ with all the users of interest. If a user mentions directly another user and both of them belong to $V$, we create an edge reflecting this interaction. The edge is assigned a weight equal to the number of mentions. To avoid sparsity of the connections, if two users of interest mention a third user, who does not belong to $V$, we create an edge between these two users. Again, the weight of this edge is the sum of mentions between the third user and the two others. Furthermore, we define a list of so-called celebrities consisting of users that have a number of unique connections exceeding a threshold $C$. We remove all connections to these celebrities since the celebrities are often mentioned by plenty of people all over the world. Mentioning a celebrity, therefore, might not be a good indication of geographical relation. The graph building procedure is depicted in Fig. 2.

A shortcoming of this method is that it can only produce an embedding for a node if that node has at least one connection to another node. Nodes without an edge can not be represented. Therefore, for an isolated node, we consider an all-zero vector as its embedding. Moreover, whenever a new node joins the graph, the algorithm needs to run again to learn feature vectors for all the nodes of the graph, making our method inherently transductive. There are some existing efforts addressing this problem. In , the authors consider a node’s embedding as a function of its natural feature; in this case the embedding could be a function of either the TF-IDF or doc2vec feature. A similar approach presented in generates a node’s embedding by sampling and aggregating features from the node’s local neighborhood. These inductive approaches will be considered in our future work.

2.4 The Timestamp Feature

3 Improvements with S2 adaptive grid

When addressing the prediction of users’ location as a classification problem, the geographical coordinate assigned to a user with unknown location equals the centroid of the class, which has been predicted for the user. A straightforward way to form the classes is taking administrative boundaries such as states, regions or countries. Such an approach brings large distance errors if the respective areas are large. Intuitively, the prediction accuracy could be improved if we increase the granularity level by defining classes that correspond to smaller areas. The tiling should also consider the distribution of users; very imbalanced custom classes should be avoided, otherwise, the training process will not be efficient. Therefore, finding an appropriate way to subdivide users into custom small geographical areas is critical.

An early work of Roller et al. has built an adaptive grid using a k-d tree to partition data into custom classes. This partitioning, though considers the distribution of users, does not necessarily produce uniform cells at the same level. Here, we split the Twitter users in the training set into small areas called S2 cells, using Google’s S2 geometry library. This library is a powerful tool for partitioning the earth’s surface. Considering the earth as a sphere, the library hierarchically subdivides the sphere’s surface by projecting it on an enclosing cube. On each surface of the cube, a hierarchical partition is made using a spatial data structure named quad-tree. Each node on the tree represents an S2 cell, which corresponds to an area on the earth’s surface. The quad-tree used in the Google S2 geometry library has a depth of $30$; the root cell is assigned the lowest level of zero and the leaf cells are assigned the highest level of $30$. The library outputs mostly uniform cells at the same level. For instance, the minimum area of level-12 cells is $3.31$ km2 and the maximum area of these cells is $6.38$ km2.

In this work, we build an S2 adaptive grid, aiming at a balanced tiling, meaning that the defined cells (geographical areas) contain a similar number of users. For this reason, we specify a threshold $T_{\max}$, as the maximum allowed number of users per cell. We build the adaptive grid from bottom to top. First, we identify the leaves corresponding to given geocoordinates. As long as the total number of users in children nodes (cells) is smaller than $T_{\max}$, we merge these nodes together; the children nodes’ users are assigned to the parent cell, i.e., a larger geographical area. We climb the tree gradually repeating this process. If we reach a specific level, $L_{\min}$, we stop the climb in order to avoid defining cells that correspond to large geographical areas; otherwise, the prediction error would increase. Figures 3 and 4 show the subdivision of users in S2 cells for the considered datasets.

Experiments

In our experiments, we employ the following three datasets, which contain tweets coming from the United States (GeoText , UTGeo2011 ) and all over the world (TwitterWorld ).

GeoText: This is a small dataset containing more than $370.000$ tweets posted by $9475$ unique users from $48$ contiguous states and Washington D.C. during the first week of March, 2010. Tweets were filtered carefully before being put into the dataset to make sure that only relevant tweets are kept. In this dataset, the geospatial coordinates of the first message of users were used as their primary location. This was done originally by the author in and followed by other authors . The dataset was already split into the training, development and testing sets with $7580$, $1895$ and $1895$ users, respectively. For the downstream tasks, tweets from a user are concatenated making a tweet document.

UTGeo2011: This is a larger dataset which was created by the authors of . The dataset is also referred to as TwitterUS in many Twitter user geolocation publications . The dataset contains approximately $38$ million tweets sent by $449.694$ users from the US. In contrast to GeoText, this dataset is noisier, namely many tweets have no location information. To treat it similarly to GeoText, all the tweets from a specific user are concatenated into a single document; a primary location is defined as the earliest valid coordinate of the tweets. Ten thousand users are selected randomly to make the development set, and the same amount is reserved for the evaluation set. The remaining users form the training set.

TwitterWorld: This is the dataset created by the authors of . The dataset contains $12$ million tweets sent by $1.39$ million users from different countries in the world, of which ten thousand users are kept for each the development set and the testing set. Moreover, only tweets that are in English and close to a city are retained. The primary location of a user in this dataset is assigned the centre of the city where most of his tweets were sent. Different from GeoText and UTGeo2011, this dataset provides purely textual information; the timestamps of messages are not available.

The location of a user is indicated by a pair of real numbers, namely, latitude and longitude. However, classification models need discrete labels. For the datasets collected from the US, we follow to employ administrative boundaries to create the class labels. By doing so, we can consider the tasks of regional and state classification as in . We rely on the Ray Casting algorithm of to decide if a location is inside a region or state’s boundary. For the region and state boundaries, we use information from Census Divisionshttps://www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf. Also, we have employed the Google S2 library, k-means and k-d tree clusterings to partition all the geospatial datasets, making other sets of labels. This supports the task of predicting the geocoordinates. More details on the settings of the partitioning schemes and their impacts will follow in the next section.

2 Performance Criteria and Experiment Design

The proposed model for geolocation of Twitter users addresses the following tasks: (i) four-way classification of US regions including Northeast, Midwest, West and South, (ii) fifty-way classification to predict the states of users, and (iii) estimation of the real-valued coordinates of users, i.e., latitude and longitude. For the region and state classification tasks, we compare the performance of our model with existing methods, by calculating the percentage of correctly classified users, which is the accuracy. Considering the estimation of the user coordinates, we measure the distance between the predicted and the actual geocoordinates and calculate the mean and the median values over the testing dataset. The distance between the predicted and the ground truth coordinates is computed using the Haversine formula . Another common way to measure the success of coordinate estimation is to calculate the percentage of estimations with accuracy better than $161$ km; this metric, known as @$161$$161$ km $\sim$ $100$ mile, has been used in many works . It is worth noting that for the classification accuracy and the accuracy @$161$ metrics, the higher values indicate a good prediction. Conversely, achieving lower values for the mean and median distance errors is desired.

Concerning the first two classification tasks, we conduct experiments on the US Twitter datasets, namely GeoText and UTGeo2011. For predicting Twitter users’ geocoordinates, experiments are performed on the three datasets. Furthermore, it should be noted that the experiments for geographical coordinate prediction use different sets of labels created by S2, $k$-d tree and $k$-means partitioning. Also, the administrative boundaries used in task (ii) are exploited for exact geocoordinate estimation.

3 Data Pre-processing and Normalization

Before computing node2vec and TF-IDF features, a simple pre-processing phase is required. First, we tokenize the tweets and remove stop words using nltkhttp://www.nltk.org/ , a dedicated library for natural language processing. Then, we replace URLs and punctuation by special characters, which results in reducing the size of the vocabulary without affecting the semantics of tweets. Again, nltk is used for stemming in the last stage of pre-processing.

4 Parameter Settings

Our framework considers four different features and each feature requires some parameters for extraction. Extracting TF-IDF using scikit-learn requires a minimum term frequency across documents min_df. For the GeoText dataset, we choose min_df=$40$. For the UTGeo2011 and TwitterWorld datasets, because of the sheer volume of data, we set min_df=$500$ and min_df=$400$, respectively. Concerning doc2vec, we select an embedding size equal to $300$. The size of the sampling window is set to $10$.

We have built the Twitter users’ graphs for the three datasets using mentions extracted from tweet messages only as discussed in Section 3.2.3. Following , we set the celebrity connection thresholds $C$ to $5$, $15$ and $5$ for GeoText, UTGeo2011 and TwitterWorld, respectively. Table I shows graph statistics for all three datasets. We use the code provided by the authors of to obtain the node2vec feature. We choose an embedding size equal to $300$. When training the embeddings, we select the weighted graph option, which takes into account the weights of edges. Other parameters are set to default values, namely the walk length $l=80$, transition parameters $p=1$, $q=1$. The sampling window size is set to $5$.

Choosing the right hyperparameters for neural networks, which are the number and size of hidden layers, is always a challenge. In our experiments, these parameters are set empirically. We set the number of hidden layers on each individual branch to 1, namely we use hidden layers $h_{11}$, $h_{21}$, $h_{31}$ and $h_{41}$ for features TF-IDF, node2vec, doc2vec, and timestamp, respectively. Also, we connect the combination layer with the softmax layer directly without adding any layer in between. All hyperparameters can be found in Table II. We use a small value for the learning rate $\alpha=0,0001$ and regularize the weights right before the output layer only. The regularization parameter $\lambda$ is set to $0,1$. The training procedure is performed using stochastic gradient descent with the optimization algorithm ADAM as the updating rule. The consecutively non-improving performance threshold $T_{val}$ is set to $10$ for GeoText and $6$ for both UTGeo2011 and TwitterWorld datasets.

Creating S2 grids requires setting the minimum cell level $L_{min}$ and maximum number of users per cell $T_{max}$. We have experimented with different settings and reported the best result in Table IV with $L_{min}=6$, $T_{max}=500$ for GeoText, $L_{min}=6$, $T_{max}=10.000$ for UTGeo2011 and $L_{min}=7$, $T_{max}=50.000$ for TwitterWorld.

5 Results

After experimenting with different parameters, normalization techniques and feature combination strategies, we report here the best obtained results. Table III presents results for regional and state geolocation for GeoText and UTGeo2011, while for the prediction of user geographical coordinates, results are presented in Table IV.

Concerning the classification tasks, our model significantly outperforms all previous works. Successful regional classification is achieved for $76\%$ of users, while for state classification the result is $64.4\%$. By leveraging the classification strength of multiple features, the improvement in regional accuracy is $9\%$ compared to the work in . Concerning the accuracy in state classification, we achieve a greater improvement that rises to $23\%$ compared to the state of the art presented in .

The estimation of geographical coordinates of Twitter users involves experiments with two types of labels, thus two sets of experiments. In the first set of experiments, we use classes corresponding to the fifty states of the US. In the second set of experiments, we employ the S2 classes described in Section 3. As can be seen in Table IV, concerning the results obtained with state labels, the mean distance error obtained with MENET is smaller than with other methods. Likewise, the median distance error and the @$161$ accuracy are better on GeoText. However, our result with these metrics is worse on UTGeo2011. The reason being that the state boundaries ignore the geographical distribution of users. The performance of MENET is improved significantly over all criteria with S2 labels, when the definition of regions takes into account the distribution of users. In this case, Table IV shows that the proposed method outperforms existing methods in terms of mean, median distance error and @$161$ accuracy on GeoText and UTGeo2011. On TwitterWorld, the median distance error is reduced more than $200km$ compared to the result in while the result for the other metrics is comparable to the state of the art. At this point, we would like to underline that the number of employed classes is critical for the performance of our method. A larger number of classes results in smaller geographical areas, which may improve the geocoordinate prediction. However, training a model with more classes may be more difficult, thus, the classification may perform worse.

As explained in Section 3.3, an S2 adaptive grid is built using two parameters: the minimum S2 cell level $L_{min}$ and the maximum number of users per cell (region, class) $T_{max}$. As an example, the geolocation result with S2 labels presented in Table IV for GeoText is associated with the minimum cell level of $6$ and the user threshold of $500$. The number of cells and their area ($m^{2}$) will vary depending on these parameters. One may wonder if this setting is optimal or not. In this section, we present an analysis of the performance of MENET with regard to different S2 parameter settings. Concretely, we run experiments using the same hyperparameter setting of MENET on GeoText with different S2 label sets. The label sets are created by either varying the minimum S2 cell level $L_{min}$ or the user threshold $T_{max}$. The results of these experiments are shown in Tables V and VI.

We can see a clear trend in the median of the distance error from the experiments with varying $L_{min}$. When $L_{min}$ increases, meaning more regions are generated, the median of the distance error decreases monotonically to a very small value (i.e., $28$ km). The reason for this is very intuitive. S2 cells at a higher level have smaller area, and if the classification performance of MENET does not get significantly worse with more classes, a predicted location will be more likely closer to the ground truth location. This also explains the increasing trend in accuracy within $161$ km. There is no clear trend in the mean of distance error. This could be explained by the sensitivity of the mean with regard to the outliers. Even if the classification accuracy of MENET goes down slightly, it may bring huge distance errors from large area cells. This has a large impact on the mean value. On the other hand, the impact of these outliers is small on the median value.

Table VI shows that when the maximum number of users per cell $T_{max}$ increases, fewer regions are created. The decreasing trend in the mean distance errors can be explained by the better classification performance when using less classes. Moreover, the median and @$161$ remain stable within the range of $200-500$ for $T_{max}$. The reason being that the classification accuracy in this range does not change significantly. The median and the @$161$, however, are much worse with $T_{max}$ set to $100$ even when the corresponding number of regions is limited. The reason being, again, that the classification performance drops dramatically. The question arises: why is the classification accuracy so low? The reason is that splitting with this setting ignores the geographically natural distribution of data. In fact, an S2 cell at level $6$ is a good fit with the area of cities, where most of the tweets originate. If we lower the user threshold $T_{max}$ in a cell, the splitting algorithm will stop at much higher cell levels for cities where the tweet density is high, thus dividing the city area into multiple smaller regions. That explains why the classification performance is very low. Figures 5 and 6 show the subvidision of GeoText at level $6$ with different values of $T_{max}$.

5.2 Feature Analysis

The MENET architecture has the capability of exploiting multiple features according to the multiview learning paradigm. In this paper, we realize the model using four features: TF-IDF, node2vec, doc2vec and timestamp. The question, then, arises: which feature contributes the most to the discriminative strength of the model? To answer this question, we conduct additional experiments with different combinations of features. Concretely, we eliminate one type of feature from the feature set and perform experiments with the rest. This can be done by temporarily removing a branch in MENET just before the concatenation layer (see Fig. 1). For a fair comparison, we use the same parameter setting for MENET as in the experiments with the full feature set. The results from the experiments on the GeoText dataset are presented in Table VII.

Compared to the results in Table IV, it is clear that the node2vec feature is the most important. Removing this feature results in a significant reduction of MENET’s performance in terms of mean distance error ($894$ km), median distance error ($480$ km) and accuracy within $161$ km ($36.5$ %). The contribution of the doc2vec feature is also noticeable, indicating an increase of more than $100$ km in terms of mean distance error, compared to the full feature set. The other features help to improve the performance slightly as removing them results in a marginal decrease in the three performance criteria.

5.3 Performance of MENET with regard to Observed Partitioning

In Table IV, we have a notable improvement in geolocation result with MENET by using S2 labels. Using Google’s S2 geometry library is one of many ways to create label sets for our classification problem. A similar partitioning strategy to Google’s S2 library is called Hierarchical Equal Area isoLatitude Pixelization of a sphere (HEALPix) . Like the S2 library, it is able to partition the sphere into a uniform grid, and has appeared in several papers for Twitter user geolocation such as . Other examples include the use of $k$-d tree and $k$-means clustering algorithms for grouping users, thus making labels as in . In this section, we aim at investigating the performance of MENET with respect to two label creation strategies, namely $k$-d tree and $k$-means subdivisions. Our experiments are again conducted on the GeoText dataset.

Following , we create groups of users using either $k$-d tree or $k$-means partitioning. The clustering of users is based on geographical coordinates, namely latitude and longitude. For $k$-d tree subdivision, we make the root node with the bounding box that contains all user coordinates. Then, the tree is made by recursively splitting nodes, which correspond to boxes, into children nodes with straight dividing lines. The splitting takes into account the larger dimension of a node, then tries to divide all users in that node into two groups evenly. Note that we use only leaves to store users, which corresponds to classes. Therefore, the dividing lines must not go through any user’s point. The recursive splitting process stops if the number of users in a cell falls below a given threshold. Following , we set the theshold to $300$ resulting in $32$ geographical cells (i.e., classes). When using $k$-means for making classes, the number of clusters is set to $32$, and the Euclidean distance metric is used. The same hyperparameter settings are kept for MENET in these experiments.

The geolocation results on the GeoText dataset with $k$-d tree and $k$-means partitionings are shown in Table VIII. It is clear that $k$-means is better than $k$-d tree in partitioning Twitter users, in the sense that it can mitigate the geolocation errors. Concretely, using $k$-means labels reduces the mean distance error with more than $30$ km. The median distance error reduces by $50$% while the accuracy within $161$ km improves by roughly $7$%. On the other hand, the performance of MENET using S2 labels is better for all the concerned performance criteria. Also, it is worth mentioning that the performance of MENET with the $k$-means labels is close to that of S2 labels. However, the S2 partitioning is more flexible in controlling the median distance error and it is stable in creating labels compared with $k$-means.

Conclusion and Future Work

Noisy and sparse labeled data make the prediction of Twitter user locations a challenging task. While plenty approaches have been proposed, no method has attained a very high accuracy. Following the multiview learning paradigm, this paper shows the effectiveness of combining knowledge from both user-generated content and network-based relationships. In particular, we propose a generic neural network model, referred to as MENET, that uses words, paragraph semantics, network topology and timestamp information, to infer users’ location. The proposed model provides more accurate results compared to the state of the art, and it can be extended to leverage other types of available information, besides the types of data considered in this paper.

The performance of our model heavily depends on user graph features. The node2vec algorithm used in this paper is transductive, meaning the graph is built on all users. In our future work, we will focus on making the model truly inductive, meaning able to generalize to never seen users.

References