The other day we decided to take part in a Hackathon, which we, of course, won’t win. And one of the tasks was to create a dataset from the VKontakte social network. Without thinking for a long time, I told my teammates that I was taking this part on myself. But it turned out to be not so simple. I want to share with you what problems I encountered and how I solved them along the way.
"Cerebro"
Finds the target audience, downloads the user ID database. Subsequently, it can be exported and uploaded to the VKontakte advertising account as your own retargeting group. Cerebro can not only search for communities that contain the desired audience, but also collect IDs of discussion participants, community administrators, user friends, as well as by day birth. In my experience, when using Cerebro, the cost of a click is reduced by 2-4 times, depending on the topic. CTR increases due to high-quality selection of the audience to whom advertising can be shown.
Lists of VKontakte friends in the mobile version
On the website m.vk.com, which is adapted specifically for mobile browsers, you can also go to the “Friends” section, after which you should click on the “All Friends” button. In the drop-down menu, you can select one of the lists created directly by the user, after which those friends that belong to this category will be displayed. It is worth noting that here you cannot view friends from public standard lists, or add friends to various categories, or edit and delete these lists, as you can do, for example, from a computer.
Pepper.ninja
The functionality is similar to Cerebro, but there are several important features. For example, Pepper.ninja also works with Facebook, although it only searches by community audience. The service can also collect audiences from website pages where the VKontakte comment widget is installed.
How to find out VKontakte ID
- Method one. We go to the VKontakte page, focusing on the address bar of the browser. The slash ( / ) is followed by the identification number. If the user did not change anything, then this will be his identification number, and if he changed it to a beautiful combination of letters and numbers, then he will have to use other methods;
- Method two. It allows you to kill two birds with one stone - find out both the digital identifier and the letter identifier. By clicking on the avatar in the upper right corner, go to the “My Settings” tab. Scrolling down the settings, we find the “Page Address” field. The numeric code on the right is exactly the digital id assigned during VKontakte registration. To avoid forgetting the address, remember or write down these numbers. You may need them in the future - from business negotiations to expanding your circle of acquaintances;
When meeting a person, it is much easier to leave him a VK id. If you have a common last name or first name, then finding you through the search is not easy. For this reason, the ID is everything. In order not to forget the ID in numbers, write them down in a notepad. This option is also suitable for video conversations. Example: the interlocutor asks you for your VK page. It is much easier to write your ID on paper and show it on camera than to torment a person with long and tedious searches.
"Barkov.net"
Selects users based on the intersection of community audiences. Gathers people who are members of groups with similar topics. Thus, we get a base only of those who are really interested in the topic. It takes into account activity on the wall and in discussions. Like Cerebro, the service counts absolutely all likes and reposts, as well as commentators, and filters the final list for the desired request. To work with a specific post, there is the “Activity in Post” function. Collects the IDs of all the user’s friends and subscribers on VKontakte.
Lists of VKontakte friends in the application
In the official mobile application, after going to the “Friends” section, you can click on the “All Friends” button and select one of your own (that is, created personally by the user) or standard lists, after which only those friends that were previously added to this category will be displayed in the full computer version of the site. You cannot add friends to various lists or edit these categories through the mobile application.
GetTarget
The GetTarget service appeared at the end of last summer and has already received many good reviews from SMM specialists. It allows you not only to gather community audiences into groups for retargeting, but also to select from the total mass only new, still hot participants or, conversely, those who have already left the community. In your personal account interface, you can add communities, monitor your audience, and track statistics.
Why do you need an ID?
From the point of view of a social network as a program, your first and last name does not matter. Just as the names of your friends and acquaintances registered on VKontakte do not matter. For a social network, the most significant element is ID. After all, he is unique. And there can be a huge number of namesakes and namesakes. Therefore, VK forwards messages not from Masha to Seryozha, but from id34853145 to id3548256.
This method helps avoid confusion and other problems. Agree, it would not be very pleasant if your message was mistakenly sent not to your friend Christina, but to her namesake living in the same city.
That's all for me. By the way, here is a link to the article , as I promised. And here is a link to our Zen channel . There are a lot of cool articles about VK that you won’t find anywhere else. And here is a link to another cool channel for self-development. Anyone who wants to learn new things and develop should definitely take a look.
Related Posts
How to easily and quickly download music from VKontakte
The growth in VK user activity in 2021 exceeded expectations
How to turn off notifications on VKontakte
CloudTarget
In addition to the standard functionality for similar services, it tracks newcomers to the desired communities on the topic of interest. One of the fastest tools—parsing a database of up to 500,000 people—takes a few seconds.
The service can search for any number of communities using keywords and filters. The output includes information about the number of bots, the average number of posts per day, and the activity rate (ER).
PS To learn more about how to turn a community into an effective online marketing channel or become a target pro, sign up for the online intensive “SMM Manager”.
The opinions of the author and the editor may not coincide. Want to write a column for Netology? Read our terms of publication. To keep abreast of all the news and read new articles, join the Netology Telegram channel.
We receive a list of members of the VKontakte community of a certain gender and age
I was inspired to create this article by the publication “Getting vk.com community members in seconds.” My article was written by a beginner and reflects experience in solving one problem. The main purpose of writing this article for me is to collect opinions, feedback and criticism of the applied approach from more experienced colleagues. In addition, I hope that the information provided here will be useful to someone.
Not long ago, in one of the test tasks for the vacancy of a junior PHP programmer, I came across a simple, but interesting task for me.
“Make a script in PHP that returns a list of VKontakte user ids, separated by newline characters, who are men over 25 years old and belong to the group vk.com/habr.” Access to information from the VKontakte database is carried out using the VK API. It’s better to start getting acquainted with the VK API with the official documentation. In order to call the VKontakte API method, you need to make a POST or GET request via the HTTPS protocol to the following URL:
api.vk.com/method/METHOD_NAME?PARAMETERS&access_token=ACCESS_TOKEN, where METHOD_NAME is the name of the method from the list of API methods, PARAMETERS are the parameters of the corresponding method, ACCESS_TOKEN is the access key.
In our task we use the groups.getMembers method, which returns a list of community members. All method parameters are described in the documentation. The method does not require an access key. In the standard form, the response comes in the form of a JSON file. You can get data from no more than 1000 users in one request. To see the output of the method live, just enter a simple request in the address bar of your browser: api.vk.com/method/groups.getMembers?group_id=habr.
We get a JSON structure with the total number of members of the vk.com/habr community and the first thousand ids in the default list sorted in ascending order.
According to the conditions of the problem, we need to display the id of users of a certain gender and age. The obvious way is to use VK API queries to select group users along with their data on gender and age, and then analyze them in PHP code and display only the necessary ones. Another possible method - the execute method - allows you to pass a script in a special VKScript language in one request to manipulate data on the server and return the already processed data. I’ll say right away that I was unable to solve the problem using the execute method. Maybe someone will indicate such a solution in the comments.
Let's take the first path. The groups.getMembers method can return the user's gender using the sex value of the fields parameter, but it does not return age. Instead, the fields parameter has a bdate field - date of birth. In addition, in queries we select a thousand users each, which means that each subsequent query should return the next thousand. For this, there is an offset parameter, which shows at what position to start sampling. We will also indicate the API version in the request.
As a result, the request will look something like this: https://api.vk.com/method/groups.getMembers?group_id=habr&offset=0&fields=sex,bdate&version=5.27
To get a file via a link, PHP has the file_get_contents() function. It gets the content by reference and returns it as a string. Please note that in order for file_get_contents() to understand the HTTPS protocol, openssl support is needed in the web server.
Then the resulting JSON content can be converted into an array using the json_decode() function. The array will contain both id and gender. The date of birth may not be indicated at all. If the date of birth is still indicated, all that remains is to obtain the age from the date of birth.
Dates of birth in bdate are stored in format strings DD.MM.YYYY if the year of birth is specified, or DD.MM if the year of birth is not specified. To find out what format the string is actually in, I used the first thing that came to mind: count(explode(“.", $user_array['bdate'])) equals 2 or 3. This method works and I don’t think it’s the narrowest script place.
To calculate age by date of birth, I found the formula hashcode.ru/questions/137939#137940. The strtotime() function understands the format of the bdate field.
We check gender and age.
If they satisfy the condition, we print id. All code in PHP
// Request package number $packet = 0; // Request packet size $limit = 1000; do { // We start each request where we left off in the previous request. $offset = $packet * $limit; // Execute the request. // The result is a JSON file with the total number and user data. // For file_get_contents() to work with https, the openssl module // must be active on the apache web server. $contents = file_get_contents("https://api.vk.com/method/groups.getMembers?group_id=habr&offset=$offset&fields=sex,bdate&version=5.27") // Convert JSON to an array $members = json_decode($contents, true); // User data is stored in the users subarray. // Each users element is an associated array with data. foreach ($members['response']['users'] as $user_array) { // If the user specified a date of birth and the user is male... if ((isset($user_array['bdate'])) && ($user_array[ 'sex'] == 2)) { // … and if the date of birth has three components (DD.MM.YYYY)… if (count(explode(“.”, $user_array['bdate'])) == 3 ) { // then we calculate the age (I found the formula on the Internet) $age = floor((time()-strtotime($user_array['bdate']))/(60*60*24*365.25)); // If the age suits us, display the user id with a newline if ($age > 25) { echo $user_array['uid'] . ""; } } } } // Move on to the next package. $packet++; } while ($members['response']['count'] > $offset + $limit); This option works great for relatively small groups, but for groups of more than 100 thousand subscribers the script does not work completely - at some point for some reason the error “file_get_contents(...): failed to open stream: Connection timed out in … on line” appears ..." I tried increasing the script execution time and the web server timeout - it didn’t help. I couldn't find a pattern.
Then another option was found - to use cURL to download the request response. To apply this method, you need to install the libcurl library in the OS, for example, in Ubuntu -
sudo apt-get install libcurl3 and enable cURL support in PHP, for example, in Ubuntu - sudo apt-get install php5-curl Now you can open a curl session in a PHP script with the curl_init() function, set connection parameters (including URL) with the function curl_setopt() and download the contents of JSON files into a string using the curl_exec() function. Then you should close the session - curl_close(). The rest of the code remains unchanged:
All code with cURL in PHP
// Request packet number $packet = 0;
// Request packet size $limit = 1000; // Initialize cURL. // To work with cURL, the libcurl library // must be installed and cURL support in PHP must be enabled. $ch = curl_init(); do { // We start each request where we left off in the previous request. $offset = $packet * $limit; // Request parameters curl_setopt($ch, CURLOPT_URL, "https://api.vk.com/method/groups.getMembers?group_id=habr&offset=$offset&fields=sex,bdate&version=5.27"); curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // Execute the request. // The result is a JSON file with the total number and user data. $content = curl_exec($ch); $members = json_decode($contents, true); // User data is stored in the users subarray. // Each users element is an associated array with data. foreach ($members['response']['users'] as $user_array) { // If the user specified a date of birth and the user is male... if ((isset($user_array['bdate'])) && ($user_array[ 'sex'] == 2)) { // … and if the date of birth has three components (DD.MM.YYYY)… if (count(explode(“.”, $user_array['bdate'])) == 3 ) { // then we calculate the age (I found the formula on the Internet) $age = floor((time()-strtotime($user_array['bdate']))/(60*60*24*365.25)); // If the age suits us, display the user id with a newline if ($age > 25) { echo $user_array['uid'] . ""; } } } } // Move on to the next package. $packet++; } while ($members['response']['count'] > $offset + $limit); // Close cURL curl_close ($ch); As I said, I think the execute method approach is possible, but I have not yet been able to get a satisfactory result in this direction.
PS Please do not think that I want to get a solution to a test task from the Habr audience. I sent the above options a long time ago and received a response. I just spent a lot of time on this task and would like to know if I was moving in the right direction and what other approaches could be used.
