This manual elaborates the relevant points of mining farm operation and maintenance. The operation and maintenance staff should know and master the basic maintenance of miners, including system upgrade, reset and TF card recovery, as well as simple fault judgments such as temperature protection, fan failure, network failure, etc. In addition, they should also have a certain understanding of power supply, network, environment, etc. of the mining farm to make corresponding inspections regularly, so that they can find and solve problems in time to reduce failure rate of miners.
This manual introduces the operation and maintenance of cryptocurrency mining farms from the following aspects: Routine Inspection, Notices for Going on and off Racks, Miner Troubleshooting.
The mining farm of Antminer will be used as an example in the manual. However, this manual is also applicable to other types of miners.
Routine Inspection
The first part of the Operation and Maintenance Manual of Cryptocurrency Mining Farms is Routine Inspection, including: Hashrate Inspection, Power Inspection, Network Inspection, etc.
Ⅰ. Hashrate Inspection
- Check the hashrate every two hours through the batch management software BTC Tools or login to websites of corresponding mining pools. BTC Tools can set mining workers in batch and view real-time hashrate and temperature. It is recommended to install Google Chrome and set it as the default browser so that you can directly double-click the scanned miner to enter the mining status page without filling in username and password. When the password of the miner has been changed so that you must fill in username and password, you can just click “Setting” in the upper right corner of the BTC Tools and change to new password. To download BTC Tools, please click here: https://pool.btc.com/tools
- Regularly check the mining worker information for possible errors or omissions, the information of the main mining pool as well as other two spare pools should all be complete. It is recommended to change the default password and do not download unauthorized firmware and overclocking firmware for upgrading to avoid illegal tampering by malicious software. The passwords can be modified in batches by APMinerTool, which can also scan and manage the miners. Notice: Only the miners in normal operation with hashrate can be scanned. To download APMinerTool, please click here: https://www.antpool.com/download.htm?m=tools
- If the miners with abnormal hashrate are found, the problems should be confirmed and resolved in time. If the problems cannot be resolved onsite, go off the shelf for repairment or return to manufacturer for maintenance.
Ⅱ. Power Inspection
- Use multimeter every day to check whether the voltage of miner PDU socket is within the normal range (Normal power value differs in specific countries, and in China it’s 220V±10%). Excessively high or low voltages will cause unstable operation or machine failure, even power supply or miner damages. If the voltage is abnormal, notify the electrician in time for maintenance.
- Check three-phase current meter in power distribution cabinet every day. If the deviation is more than 15%, notify the electrician to check whether the three-phase load is balanced. The unbalance may cause higher voltage of a certain phase. It is necessary to distribute the miners as evenly as possible into the three-phase circuit instead of the same phase circuit.
- Regularly check whether the miners have electric leakage. The cases of some miners may have electric leakage due to poor ground wire, which may cause the control board and hashboard to be damaged during the miner operation. The electrician is needed in time to deal with the poor grounding.
Detection of Electric Leakage:
- Method 1: When the miner is running, use multimeter to test whether there is voltage on the miner case and the bare metal shelf. If there is voltage, it means the grounding is poor (this method is only suitable for the situation that the miner case doesn’t touch the metal shelf)
- Method 2: When the miner is running, measure whether there is voltage on the metal cover of miner network port and the miner case. If there is voltage, it means the grounding is poor (This method is only suitable for the situation that the metal cover of miner network port doesn’t touch the control board.)
Note: The measured voltage should be less than 1V, otherwise the grounding should be done.
- Regularly check the ground wire of farm shelf and miner socket (grounding resistance <4Ω). If it is not grounded or the grounding is not reliable, notify professional electrician for solution in time. During the operation and maintenance process, if the case of miner power supply feels electrically numb, it is necessary to check the ground wire.
- Regularly check the aging of wires and cables, use infrared thermometer to check the temperature of breaker, cables, copper flat cables and connectors. If the temperature of copper flat cables or copper nose is over 70℃, notify the electrician in time for solution.
- The mining farm should avoid frequent power outage. The power outage should be planned in advanced and start first from the lowest air-switch on the shelf, till the master switch is turned off step by step. When the power supply is resumed, ensure all the miner switches on the shelf have been off and turned on from the master switch step by step to avoid the miners damaged by voltage surge.
Ⅲ. Network Inspection
- Regularly check the spare network on the router or the laptop connected to the spare network. The networks for mining and routine life need to be separated to avoid the network risks from unidentified websites as well as hashrate fluctuation due to network speed affected by downloading and watching video. If the network needs to be shared in special cases, the network administrator should limit the IP address of the living area on the router or switch to avoid access to unidentified websites.
- Regularly Ping the mining pools used by current miners. If the delay is more than 100ms, the internal and external networks need to be checked. The LAN can be checked by the network administrator to confirm the status of network device. The equipment and lines of external network should be checked by the ISP. Big delay of the network will cause hashrate fluctuation. The Ping function is on the Network-Diagnostics page. Fill in the mining pool URL used by current miner and click Ping to view the delay time. To check network in the case of zero hashrate, Ping without accessing to current mining pool proves network failure.
- Check the temperature of network equipment every day. If the temperature on equipment case is over 45℃, it’s necessary to change the position or install a fan for heat dissipation. Do not put the network equipment and cables near the air outlet of miners. If the CPU temperature is too high, the stability will be affected and even crash will occur. The main network cable should be of good quality. Turn off the miner if the power outage is more than 15 minutes. The power consumption and hashrate are interrelated. If the hashrate is insufficient or zero, the power consumption will be reduced accordingly.
Ⅳ. Temperature and Humidity Inspection
- Measure the temperature of the miner air inlet every day and try to control it between 10℃ to 25℃. If the temperature near the miner air inlet is over 30℃, it will easily trigger high temperature protection with zero hashrate, so it is necessary to reduce the ambient temperature in time. Check whether there’s warm air reflow near the miner. If the small fans of some third-party power supply and the miner fan are in opposite direction, the temperature will be overlapped, then the directions need to be adjusted. The miner hardware under high temperature protection is not damaged, and the hashrate can be resumed when the ambient temperature is lowered. The mining farms need to make plans to reduce the ambient temperature as soon as possible, such as water curtain, negative pressure fans, thermal isolation (use baffle and other materials to stop the warm air backflowing to the air inlet area).
- Measure the humidity near the miner air inlet every day. Check the shelf, miners, power cords and other locations for condensation. Reduce the humidity if it is over 65%. If the water curtain and the shelf is too close, the humidity will also increase. Generally, the distance should be maintained 2 to 3 meters, not too far or too close (longer distance will reduce the cooling effect). Excessive humidity for long and poor air quality due to large chemical plants nearby will accelerate the corrosion or even damage of the hashboard. The amount of curtain water and the air inlet speed can be reduced as needed. It is recommended not to run the miners near the saline-alkali area or coastal area, because the miners will be easily damaged by oxidation or corrosion.
- Check the water curtain every day for abnormal water leakage to the ground inside the farm, whether the water flows evenly and whether it’s blocked by white alkaline residue. The blocked water curtain or uneven water flow will reduce the cooling effect, so the water curtain should be cleaned regularly. The water level and the pump should also be checked regularly.
- In the area extremely cold in winter, the heating fan should be turned on in time.
Ⅴ. Environment Inspection
- Check the running status of negative pressure fan every day. The mining farm needs to be cleaned to ensure free from dust and paper scraps.
- Regularly check the miners for dust every month. The front fan can be removed for inspection. If a large amount of dust, catkins, etc. are found near the heat sink, clean up in time to prevent high temperature accumulation. Use anti-static soft brush to gently brush off the dust. Be careful not to use hard brush or exert too much force, otherwise the heat sink may fall off. Blow off the dust after it’s being brushed.
- For the mining farms in heavy sands and dusts, check dust mesh every week and clean with tools such as vacuum cleaner regularly. For the farms in heavy dusts and sands without installing protective nets, it is necessary to take appropriate measures in time to prevent dusts.
- Prevent rains or water drifting in farms to damage miners, especially the doorway and some unclosed areas.
Ⅵ. Visual Inspection
- Check miner air inlet every day to ensure free from garbage and paper scraps. If any, remove in time to ensure air volume to avoid zero hashrate due to high temperature protection.
- Check miner indicators every day and take measures in case of red(fault) light. Check the running of two fans in the miner. If the fan blades are damaged or out of work, replace new one in time. The running status of fans can also be checked in Kernel log on the mining status page. Make sure the fans are installed in correct direction, otherwise the hot air cannot be exhausted out of the miner.
- Check whether the network cable is damaged, out of service or abnormally overheated. It is not recommended to use power splitter because it is easy to trigger overload. For 220V power cord, it is recommended to select the one with 10A or above with a length of less than 1.5m.
Notices for Going on and off Racks
The first part of the Operation and Maintenance Manual of Cryptocurrency Mining Farms is Notices for Going on and off Racks, including: Notices for Going On Racks and Notices for Going Off Racks.
Ⅰ. Notices for Going On Racks
- Ensure no violent collisions on the surface of the miner, shake to ensure no heat sink falling out and check external appearance of two fans.
- Before going on racks, check whether the flat cable (fan, power supply, hashboard) has been properly plugged, the 6pin cable connecting to hashboard shall be plugged in correct direction. The power cord must be properly plugged on the control board, otherwise the hashboard will be damaged. Wrong direction of power cord plugged on the control board will also result in damages. Loose plug or wrong direction may make the miner burned. Loose power cord in socket may cause poor contact, so it’s important to select high-quality sockets and power cords.
- Move the miners carefully and gently. Do not directly hold hashboard or power cord by hand. Do not throw from a height arbitrarily. When asking porters to move the miners, it’s necessary to tell them to handle with great care, otherwise the heat sinks and chips on the hashboard will fall out or the fans will be damaged.
- When miners are set in racks, use batch software APMinerTool or BTC Tools to modify the mining workers and set the static IP as needed, confirm whether the hashrate is normal. When putting single miner on the racks, set the IP address as its original shelf position.
- When deploying miners in batches on racks, properly plan the IP addresses for convenience of subsequent static IP addresses, DHCP should be separated from static IP. For example, if there are 100 miners to be assembled on the racks, the gateway can be set as 172.16.1.254 and DHCP can be set as 172.16.1.101-240, the reserved IP (1-100) can be used for subsequent static IP address. Under normal circumstances, one shelf is managed by one network segment. To set the static IP address in batches, use APMinerTool tool and click on the tab “Configure IP”.
Ⅱ. Notices for Going Off Racks
- Reconfirm whether the miner fault can be eliminated on site before taking it off the racks. Some problems such as high temperature protection and fan failure (mining farms can buy some fans as backup), network failure and system failure can be solved on site. Try not to return the normal machines due to misjudgment. Fan failure can be viewed in the kernel log by pulling the bar to the bottom on the mining log page.
- Confirm whether IP address corresponds to the miner to avoid taking by mistake. If the miner IP cannot be confirmed, use software IP Reporter, click START and press Miner IP button or click “Setting IP” in APMinerTool and click the IP button on the miner to confirm.
- Miners must be powered off before going on and off the racks and be handled carefully. Do not hold flat cable or power cord of hashboard directly. Do not throw from a height arbitrarily.
- If there’s thermal isolation on the shelves in the farm, the empty holes where miners have been removed should be blocked to avoid warm air reflux. The miners taken off racks should have dust and moisture proof treatments and be placed in designated area in order before onsite requirement or returning to manufacturer for maintenance.
Miner Troubleshooting
There are many reasons for the miner fault, so we should start from the simple solutions, such as network inspection, firmware upgrading, resetting, cable inspection, power supply replacement, dust cleaning, reducing ambient temperature, etc. If the problems haven’t been solved yet, we should check from other aspects, such as power and environment. If the problems cannot be solved on site, return to manufacturer for maintenance.
For troubleshooting and solutions for Antminer, please refer to related articles of EastShore:
- Troubleshooting in ANTMINER GENERAL MINING GUIDE | EastShore Mining Devices
- Troubleshooting for Antminer: hashing board | EastShore Mining Devices
- Troubleshooting for Antminer: controller & fan | EastShore Mining Devices
- Troubleshooting for Antminer: Common External Problems | EastShore Mining Devices
- Miner Virus Prevention and Solutions | EastShore Mining Devices
For faults and solutions of other types of miners, please consult with the manufacturers.
Appendix 1
Ⅰ. Daily Operation and Maintenance Inspection Table
The following items need to be inspected in daily operation and maintenance and relevant solutions need to be taken to reduce the failure rate. The PDF version of the Daily Operation and Maintenance Inspection Table could be downloaded here: https://github.com/eastshoremining/Others/tree/master/MiningFarmsMaintenance
Ⅱ. Miner Troubleshooting Table
A brief Miner Troubleshooting Table is shown as follows. The PDF version of the table could be downloaded here: https://github.com/eastshoremining/Others/tree/master/MiningFarmsMaintenance
Appendix
Series articles on crypto miner maintenance:
- Precautions for Cryptocurrency Miner | EastShore Mining Devices
- Professional Miner Maintenance suggestions from Antminer technicist | EastShore Mining Devices
- Operation and Maintenance Manual of Cryptocurrency Mining Farms | EastShore Mining Devices
- Miner Virus Prevention and Solutions | EastShore Mining Devices
- Practical Guide for Mining Noise Control | EastShore Mining Devices
- Practical Guide for Miner Cooling | EastShore Mining Devices
