Thứ Năm, 30 tháng 5, 2024

PHP get data from external url encoding UTF-8

If you want loading a HTML from an external server. It doesn’t matter if you want to get content from page with meta charset=”utf-8″, we use file_get_contents() function normally. But if it is not utf-8, another code like Shift_JIS or any other. You will get something similar to this:

 

Get content error format

Get content error format

 

I tried both saving the HTML to a file and outputting it with UTF-8 encoding. It didn’t work so it means file_get_contents() is already returning broken HTML.

This doesn’t mean file_get_contents() doesn’t work or the site blocks content. The main problem is in the charset tag [charset=Shift_JIS]. To deal with this, I use mb_detect_encoding (Detect character encoding) and iconv (Convert string to requested character encoding) functions. I for example with Japanese font format, after encode you will get the following result:

Meta charset Shift_JIS

Meta charset Shift_JIS

Note:

– If you need to replace the image link, css, js,… you can use the str_replace() function

– If you are selective about some content, use the preg_match() function

Eg:

$url = ‘http://onlinemeetingsoft.com/jp/cisco-webex-meetings-download.html’;
$htmlContent = file_get_contents($url);

$currentString = [‘<img src=”../../path/’];
$replaceString = [‘<img src=”http://your_url/’];
$newContent = str_replace($currentString, $replaceString, $htmlContent);

// handle with newContent…

// If you want to get the content in this table (table in td): ‘<table><tr><td><table border=0 cellpadding=2 cellspacing=0 align=center’

preg_match(‘/<\/table><tr><td><table border=0 cellpadding=2 cellspacing=0 align=center(.*?)<\/table>/s’, $htmlContent, $match);

// var_dump($match);
// echo $match[0];

/**
* Encoding
*/
function encoding($string) {
$currentEncoding = mb_detect_encoding($string, ‘auto’);
$result = iconv($currentEncoding, ‘UTF-8’, $string);
return $result;
}

echo encoding($match[0]);

 

Result PHP get data from external url encoding UTF-8:

 

 

Tags: 

Thứ Tư, 10 tháng 8, 2022

Download Font ABC

Chúng ta đều biết sau khi muốn sử dụng các font trên máy tính thì việc đầu tiên là hay cài font UnicodeFont Vni hay Font ABC(VnTime, TCVN3) hoặc thêm vài dạng font đẹp mang tính trang trí như font chữ Thư pháp, font chữ Hùng Lân… chẳng hạn.

 

download-font-abc-fontchudepvn

 

Khi cài font chữ vào máy tuy đơn giản nhưng bạn cũng nên chú ý chỉ nên cài những font chữ cần thiết, tùy nhu cầu sử dụng của chúng ta mà tìm để cài vào máy. Việc này sẽ khiến cho hệ điều hành trở lên hoạt động nhẹ nhàng hơn, khi chọn font chữ để sử dụng cũng dễ dàng hơn. Dưới đây là link tải font ABC với các lựa chọn để các bạn sử dụng nhé.

Download Font ABC:

– Tải trọn bộ font ABC (2,5MB)

– Tải bộ font chữ thường ABC (1,5MB)

– Tải bộ font chữ hoa ABC (1MB)

Some useful MySQL query commands

 This article lists some MySQL commands that may be helpful for you.

I. MySQL commands

1. COUNT and GROUP BY

If you want to display a list of how many tutorials are written by each user, you can use COUNT and GROUP BY.
For example, we have the Blogs table as follows:

 

mysql-query-commands-useful-1

 

SELECT operator_id, count(*) as count FROM blogs group by operator_id

Result:

mysql-query-commands-useful-2

 

2. MySQL queries many diferent databases on the same servers

If the query combines 2 Tables on 2 Different Databases (on the same server), everything is very easy with the following command:
Example:

SELECT a.*, b.* FROM SchemaA.blogs as a
LEFT JOIN SchemaB.blogs as b ON a.category_id = b.category_id

 

3. MySQL root password change

Run in MySql editor:

SET PASSWORD FOR ‘root’@’localhost’ = PASSWORD(‘mypass’);
FLUSH PRIVILEGES;

 

mysql-change-root-password

MySQL root password change

 

4. MySQL LEFT JOIN with condition:

We have the following 2 tables:

– tblUsers: user_id, user_name
– tblUserHistories: user_id, date_in, date_out

SELECT a.*, b.*
FROM tblUsers as a
LEFT JOIN tblUserHistories as b
ON a.user_id = b.user_id
AND
a.user_id = (SELECT user_id FROM tblUserHistories WHERE user_id = m1.user_id ORDER BY id DESC LIMIT 1)

WHERE
b.date_in >= ‘2021-01-01’
ORDER BY
b.date_in ASC, a.user_name ASC

 

5. How can I find all the tables in MySQL with specific column names in them?

SELECT DISTINCT TABLE_NAME
FROM INFORMATION_SCHEMA.COLUMNS
WHERE COLUMN_NAME LIKE ‘%file_name%’
AND TABLE_SCHEMA = ‘database_name’

 

II. MySQL multiple databases different servers

– A remote server with a database table, the table type of the remote table may be any type supported by the remote mysqld server, including MyISAM or InnoDB.

– A local server with a database table, where the table definition matches that of the corresponding table on the remote server. There is no data file on the local server. Instead, the table definition includes a connection string that points to the remote table.

mysql-query-commands-federated-structure

 

To create a FEDERATED table you should follow these steps:

1. Create the table on the remote server. Alternatively, make a note of the table definition of an existing table, perhaps using the SHOW CREATE TABLE statement.

2. Create the table on the local server with an identical table definition, but adding the connection information that links the local table to the remote table.

For example, you could create the following table on the remote server:

CREATE TABLE test_table (
id     INT(20) NOT NULL AUTO_INCREMENT,
name   VARCHAR(32) NOT NULL DEFAULT ”,
other  INT(20) NOT NULL DEFAULT ‘0’,
PRIMARY KEY  (id),
INDEX name (name),
INDEX other_key (other)
)
ENGINE=MyISAM
DEFAULT CHARSET=utf8mb4;

 

To use the first method, you must specify the CONNECTION string after the engine type in a CREATE TABLE statement.
For example:

CREATE TABLE federated_table (
id     INT(20) NOT NULL AUTO_INCREMENT,
name   VARCHAR(32) NOT NULL DEFAULT ”,
other  INT(20) NOT NULL DEFAULT ‘0’,
PRIMARY KEY  (id),
INDEX name (name),
INDEX other_key (other)
)
ENGINE=FEDERATED
DEFAULT CHARSET=utf8mb4
CONNECTION=’mysql://fed_user@remote_host:9306/federated/test_table’;

The format of the connection string is as follows:

scheme://user_name[:password]@host_name[:port_num]/db_name/tbl_name

Where:

– scheme: A recognized connection protocol. Only mysql is supported as the scheme value at this point.
– user_name: The user name for the connection. This user must have been created on the remote server, and must have suitable privileges to perform the required actions (SELECT, INSERT, UPDATE, and so forth) on the remote table.
– password: (Optional) The corresponding password for user_name.
– host_name: The host name or IP address of the remote server.
– port_num: (Optional) The port number for the remote server. The default is 3306.
– db_name: The name of the database holding the remote table.
– tbl_name: The name of the remote table. The name of the local and the remote table do not have to match.

 

Refer: https://softdownloadfree.com/some-useful-mysql-query-commands

Chủ Nhật, 7 tháng 8, 2022

Move large mysql database to new server

We all know the importance of a database in every system, in this article we ask you to back up, move database to another server.

This will be easily handled if your db has a small number of records. And if the number of records is too large (a few million records or more) then exporting on the current machine and importing to another server will take a lot of time, not feasible.

You can use one of two ways:

Method 1:

mysql -u root -p your_db_name < your_path/../database.sql
Example: C:/Users/admin/dumps/backup-2020.sql

 

Method 2: login with root:

mysql -u root -p –binary-mode=1
use your_db_name
source your_path/../database.sql (C:/Users/admin/dumps/backup-2020.sql)

 

However, in many cases there will be a dump error, because of the data type, it is not possible to import all the records into the new database.

I assume you are using mysql with Xampp server service, the data when running is saved with the folder name as the database name located C:\xampp\mysql\data\your_db_name, at the new server address, also installed corresponding xampp.

You just need to copy the folder your_db_name, move to the new server, the backup will be successful, together with some log files in mysql, the specific steps are as follows:

 

move-large-mysql-database-to-new-server

Your path database name

 

Step 1: Stop Mysql, Apache if you have started the application at the new server.

Step 2: Backup 3 files: ib_logfile0, ib_logfile1, ibdata1 in the new server.

Step 3: In the root server, at address C:\xampp\mysql\data

– Copy the folder your_db_name
– Copy 3 files: ib_logfile0, ib_logfile1, ibdata1
-> Move 1 folder and 3 files above to C:\xampp\mysql\data in the new server.

Step 4: Start MySQL, Apache in the new server.

Step 5: Check the connection to the new DB.

Note: If you want to return to the new server state before backup, you can return 3 files ib_logfile0, ib_logfile1, ibdata1 server to  C:\xampp\mysql\data, you will be able to reconnect with DB (if available before).

Tags: 

Thứ Tư, 20 tháng 7, 2022

Optimize your MySQL Database Performance

MySQL is a free database management system widely used in PHP. When creating tables in MySQL there are many types of Storage Engine to choose from. MyIsam & Innodb & Memory are three popular types. Choosing a store tool is extremely important before you design the database, helping to improve query performance. If the store engine changes later, it will take a long time if the data on the table is large. Let’s talk about it.

 

optimize-your-mysql-database-performance

 

I. MySQL storage engines

 

1. MyISAM

This Storage Engine allows full-text search indexes. Therefore, this Storage Engine gives speed read and search quickly.

The downside of MyISAM is that it uses table-level locking mechanism, so when adding, editing or deleting a certain record in the same table, that table will be locked, not allowed until the previous operation done.

 

2. InnoDB

Currently InnoDB is set by default when creating DB, it supports Foreign key foreign key and since version 5.6, InnoDB also supports Full-text search indexes.

The best insert / update / delete speed because it works according to Row Level Locking mechanism, so when adding, editing or deleting a table, only the records are manipulated, and other activities on this table are not affect.

 

3. Memory

Memory engine is the most suitable engine for fast data access, as everything is stored in RAM. And so when restarting MySQL or Server, all data will be deleted.

 

So we should choose the store engine to suit the purpose of our project:

– MyISAM: For an application with high reading frequency such as news sites, blogs, you should use MyISAM

– InnoDB: For applications with high insert and update frequency such as: Forums, social networks … you should use InnoDB.

– Memory: For tables containing data that has not been stored for too long, and for user sessions, you should use Memory

II. Optimize queries to MySQL Database

 

1. MySQL insert multiple rows

You cannot run the INSERT statement in a loop, which actually affects the speed of the connection to the database in case your loop has many values.

Let’s use an INSERT statement in this case, for example:

INSERT IN TO USERS (full_name, age, gender) VALUES (‘John’, 20, 1), (‘Sara’, 30, 0), (‘Mary’, 25, 0);

 

2. MySQL multiple update in one statement

Similar to the multiple INSERT statement, the UPDATE should be optimized when you want to perform the same task.

Please use ON DUPLICATE KEY UPDATE to apply, for example:

INSERT IN TO USERS (id, full_name, age, gender) VALUES (1, ‘John’, 21, 0), (2, ‘Sara’, 31, 0), (3, ‘Mary’, 26, 1) ON DUPLICATE KEY UPDATE full_name=values(full_name), age=values(age), gender=values(gender);

3. Do not use LIMIT OFFSET with tables with large data

For example:

SELECT * FROM TABLE LIMIT 10 OFFSET 80000 (starting at 80001)
OR
SELECT * FROM TABLE LIMIT 80001, 10

Please replace it with:

SELECT * FROM TABLE WHERE id > 80000 LIMIT 10

 

4. Indexing in MySQL

– How to type index compound vs index:

+ If you frequently query for 2 fields, use composite index (a, b) rather than index (a), index (b). index (a, b) is faster
+ If the query is only 1 field, then just index a without index b

– Each individual field does not need UNIQUE, but when combined as the home address, you can type INDEX as follows:

CREATE UNIQUE INDEX index_street_number_city ON addresses (street, house_number, city);

* Note: You should only index to support the frequently read queries, the frequently used fields. The more indexes, the more it affects inserting, deleting and updating.

 

5. Use the correct field type for the data

There are many different types of fields available through MySQL, but it’s best to use those that are familiar with strings and numbers.

For example, if you only store the numbers 1, 2, and 3 in a field, better use TINYINT than INTERGER. The storage space on the server will be more saved.

 

III. Some useful MySQL query commands

 

1. The most basic thing in a SELECT query

SELECT query is not to use SELECT * if it’s not absolutely necessary.

 

2. SQL CONCAT Function:

CONCAT function is used to concatenate two or more strings to a single string.

SELECT CustomerID, CONCAT(FirstName, LastName) AS FullName, CONCAT(PostalCode, Town, District, City) AS Address FROM Customers;

Result:

SQL-CONCAT-Function

 

3. MySQL JOIN Select last record:

SELECT a.id, a.user_id, a.user_name
FROM Customers as a
LEFT JOIN ViewHistories as b ON b.user_id = a.user_id
WHERE b.id = (SELECT MAX(id) FROM ViewHistories WHERE user_id = a.user_id)
ORDER BY a.name

If the user_name includes Japanese(カタカナ), it is possible to ORDER BY condition: ORDER BY user_name collate utf8_unicode_ci

 

4. Insert record if not exists in table [duplicate]

$db = DB::getInstance();
$user_id = 1001;
$name = ‘John’;
$age = ’26’;
$birthday = ‘1999-01-01’;

$sql = “INSERT INTO users (user_id, name, age, birthday)
SELECT * FROM (SELECT ?, ?, ?, ?) AS tmp
WHERE NOT EXISTS (
SELECT user_id FROM users WHERE user_id = ?
)”;

$run = $db->prepare($sql);
if ($run->execute([$user_id, $name, $age, $birthday, $user_id])) {
// …
// ex: $lastInsertId = $db->lastInsertId();
}
if (!empty($lastInsertId)) {
return ‘This data already exists !’;
}

 

5. How to delete from multiple tables in one MySQL command?

Use a JOIN in the DELETE statement.

Example: Delete record two table (logs, log_detail) with $id = 1 in logs table

$sql = “DELETE a, b
FROM logs as a
JOIN log_details as b ON b.log_id = a.id
WHERE a.id = $id “;

 

Thank you for reading the article.


Thứ Năm, 17 tháng 3, 2022

Automatically backup MySQL database with Batch file on Windows

Data backup is always very important in the way each system operates. In Linux you can easily automatically install the Crontab command, so what about Windows? We need to write code in Batch file and setup Task Schedule in Windows operating system. Very simple and fast.

Task Schedule is available in Windows operating system. You need to turn it on and map it to a Batch file. Our problem is running on Windows, maybe Windows 7, 10, Windows server and backing up data from MySQL.

The need to prepare:

1. Assuming we are using Xampp, you need to find the path of the mysqldump file

2. Prepare information about database: User, Password, Database Name

3. Prepare a database storage path. Database in this article will be saved as folder name of database, which will contain file mysql with format Year-month-date.sql

Doing:

1. Open the task schedule by either search command or in the Control panel.

2. Create Basic Task

3. Type name of task

4. Select auto-run time: daily, weekly, monthly, …

5. By default, the Task will work immediately after the setup completes. You can pause and end the task at any time by right-clicking and selecting Disable or Delete

In Batch file:

– rem: comment your code

– set date with format Year-month-day: %yyyy%-%mth%-%dd%

– create folder BACKUPPATH: md %BACKUPPATH%

– use MYSQLDUMPFILE to export your Data

 

taskschedule-start

 

@echo off 

CLS
ECHO Date format = %date%
rem echo %DATE% %TIME%

REM Breaking down the format
FOR /f “tokens=2 delims==” %%G in (‘wmic os get localdatetime /value’) do set datetime=%%G
ECHO dd = %datetime:~6,2%
ECHO mth = %datetime:~4,2%
ECHO yyyy = %datetime:~0,4%
ECHO/

REM Building a timestamp from variables
SET “dd=%datetime:~6,2%”
SET “mth=%datetime:~4,2%”
SET “yyyy=%datetime:~0,4%”
SET “Date=%yyyy%-%mth%-%dd%”
ECHO Backup data at Date: %Date%
ECHO/

rem —– Start set path and backup —–
set DATE=%Date%
set PATH=C:\dev\xampp\htdocs\your_folder\data-backup\
set MYSQLDUMPFILE=C:\dev\xampp\mysql\bin\mysqldump
set USER=root
set PASSWORD=

set DBNAME=YOUR_DATABASE_NAME

set BACKUPPATH=%PATH%\%DBNAME%

if not exist %BACKUPPATH% md %BACKUPPATH%

“%MYSQLDUMPFILE%” –user=”%USER%” –password=”%PASSWORD%” –result-file=”%BACKUPPATH%\%DATE%.sql” “%DBNAME%”

echo Done!
rem pause
exit

 

Delete old files:

Over time, the number of backup files will gradually increase. If you want to reduce storage space, the way to do it may be to delete old backup files. Leave only the latest files.

The following example will keep the 10 newest files and delete the other old files:

echo Delete old files…

for /f “skip=10 delims=” %%a in (‘dir /a-d /o-d /b /s %BACKUPPATH%’) do DEL “%%a”

echo Done!

exit

 

Setup time:

If you set it to run automatically at 1am every Sunday, you’ll see the backup file in folder C:\dev\xampp\htdocs\your_folder\data-backup\YOUR_DATABASE_NAME

2021-01-17.sql (created 01:00 am)

2021-01-24.sql (created 01:00 am)

2021-01-31.sql (created 01:00 am)