This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package models | |
// News <model> | |
// is used to describe article model. | |
type News struct { | |
ID string `json:"_id" bson:"_id"` | |
Title string `json:"title" bson:"title"` | |
Preamble string `json:"preamble" bson:"preamble"` | |
TimeAdded int64 `json:"time_added" bson:"time_added"` | |
Link string `json:"link" bson:"link"` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package crawler | |
import ( | |
"fmt" | |
"time" | |
"github.com/gocolly/colly" | |
) | |
// SecretMag <struct> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package utils | |
import ( | |
"crypto/sha1" | |
"encoding/hex" | |
) | |
// MakeHash <function> | |
// is used to create hash from string | |
func MakeHash(s string) string { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package crawler | |
import ( | |
"time" | |
) | |
// NewsCrawler <interface> | |
// is used to describe news crawler class instance | |
type NewsCrawler interface { | |
Run() []models.News |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package db | |
import ( | |
"context" | |
"fmt" | |
"time" | |
"go.mongodb.org/mongo-driver/mongo" | |
"go.mongodb.org/mongo-driver/mongo/options" | |
"go.mongodb.org/mongo-driver/mongo/readpref" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package db | |
// FillSeedsInformation <function> | |
// is used to fill information that needed before usage of news endpoint | |
func (s Service) FillSeedsInformation() { | |
s.fillNewsSources() | |
s.fillNewsTypes() | |
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package db | |
import ( | |
"go.mongodb.org/mongo-driver/bson" | |
) | |
const ( | |
newsSourcesCollection = "news_sources" | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package db | |
import ( | |
"go.mongodb.org/mongo-driver/bson" | |
"go.mongodb.org/mongo-driver/mongo/options" | |
) | |
const ( | |
newsCollection = "news" | |
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package server | |
import ( | |
) | |
// Init <function> | |
// is used to initialize server and all the corresponding services such as DB, Utils, Workers | |
func Init() { | |
// utils |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
package crawler | |
import ( | |
"time" | |
) | |
// NewsCrawler <interface> | |
// is used to describe news crawler class instance | |
type NewsCrawler interface { | |
Run() []models.News |